Inherent Challenges of Post-Hoc Membership Inference for Large Language Models

arXiv:2406.17975v1 Announce Type: cross
Abstract: Large Language Models (LLMs) are often trained on vast amounts of undisclosed data, motivating the development of post-hoc Membership Inference Attacks (MIAs) to gain insight into their training data composition. However, in this paper, we identify inherent challenges in post-hoc MIA evaluation due to potential distribution shifts between collected member and non-member datasets. Using a simple bag-of-words classifier, we demonstrate that datasets used in recent post-hoc MIAs suffer from significant distribution shifts, in some cases …

arxiv attacks challenges cs.cl cs.cr cs.lg data development evaluation identify insight language language models large llms training training data vast