Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning

arXiv:2402.12168v1 Announce Type: new
Abstract: Recently, various parameter-efficient fine-tuning (PEFT) strategies for application to language models have been proposed and successfully implemented. However, this raises the question of whether PEFT, which only updates a limited set of model parameters, constitutes security vulnerabilities when confronted with weight-poisoning backdoor attacks. In this study, we show that PEFT is more susceptible to weight-poisoning backdoor attacks compared to the full-parameter fine-tuning method, with pre-defined triggers remaining exploitable and pre-defined targets maintaining high confidence, even …

application arxiv attacks backdoor backdoor attacks cs.ai cs.cr defending fine-tuning language language models parameter poisoning question security strategies study updates vulnerabilities