Seeing Is Not Always Believing: Invisible Collision Attack and Defence on Pre-Trained Models

arXiv:2309.13579v2 Announce Type: replace
Abstract: Large-scale pre-trained models (PTMs) such as BERT and GPT have achieved great success in diverse fields. The typical paradigm is to pre-train a big deep learning model on large-scale data sets, and then fine-tune the model on small task-specific data sets for downstream tasks. Although PTMs have rapidly progressed with wide real-world applications, they also pose significant risks of potential attacks. Existing backdoor attacks or data poisoning methods often build up the assumption that the …

arxiv attack bert big collision cs.ai cs.cr data data sets deep learning defence gpt great large paradigm scale task train