User Inference Attacks on Large Language Models

arXiv:2310.09266v2 Announce Type: replace
Abstract: Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and …

applications arxiv attacker attacks called cs.cl cs.cr cs.lg data end fine-tuning language language models large llms privacy study threat threat model user data