Teach LLMs to Phish: Stealing Private Information from Language Models

arXiv:2403.00871v1 Announce Type: new
Abstract: When large language models are trained on private data, it can be a significant privacy risk for them to memorize and regurgitate sensitive information. In this work, we propose a new practical data extraction attack that we call "neural phishing". This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data with upwards of 10% attack success rates, at times, …

adversary arxiv attack call can cs.ai cs.cl cs.cr cs.lg data extraction information language language models large llms phish phishing privacy privacy risk private private data risk sensitive sensitive information stealing teach work