To Trust or Not To Trust Prediction Scores for Membership Inference Attacks. (arXiv:2111.09076v2 [cs.LG] UPDATED)

Membership inference attacks (MIAs) aim to determine whether a specific
sample was used to train a predictive model. Knowing this may indeed lead to a
privacy breach. Most MIAs, however, make use of the model's prediction scores -
the probability of each output given some input - following the intuition that
the trained model tends to behave differently on its training data. We argue
that this is a fallacy for many modern deep network architectures.
Consequently, MIAs will miserably fail …

attacks lg prediction trust