Epsilon*: Privacy Metric for Machine Learning Models

We introduce Epsilon*, a new privacy metric for measuring the privacy risk of a single model instance prior to, during, or after deployment of privacy mitigation strategies. The metric requires only black-box access to model predictions, does not require training data re-sampling or model re-training, and can be used to measure the privacy risk of models not trained with differential privacy. Epsilon* is a function of true positive and false positive rates in a hypothesis test used by an adversary …

access box can cs.cr cs.ds cs.lg data deployment instance machine machine learning machine learning models measure measuring metric mitigation mitigation strategies predictions privacy privacy risk risk single strategies training training data