PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps

arXiv:2403.09562v1 Announce Type: new
Abstract: The pre-training and fine-tuning paradigm has demonstrated its effectiveness and has become the standard approach for tailoring language models to various tasks. Currently, community-based platforms offer easy access to various pre-trained models, as anyone can publish without strict validation processes. However, a released pre-trained model can be a privacy trap for fine-tuning datasets if it is carefully designed. In this work, we propose PreCurious framework to reveal the new attack surface where the attacker releases …

access arxiv can community cs.cr easy fine-tuning language language models offer paradigm platforms privacy processes standard training turn validation