Hijacking Large Language Models via Adversarial In-Context Learning

arXiv:2311.09948v2 Announce Type: replace-cross
Abstract: In-context learning (ICL) has emerged as a powerful paradigm leveraging LLMs for specific downstream tasks by utilizing labeled examples as demonstrations (demos) in the precondition prompts. Despite its promising performance, ICL suffers from instability with the choice and arrangement of examples. Additionally, crafted adversarial attacks pose a notable threat to the robustness of ICL. However, existing attacks are either easy to detect, rely on external models, or lack specificity towards ICL. This work introduces a …

adversarial adversarial attacks arxiv attacks context cs.cl cs.cr cs.lg examples hijacking instability language language models large llms paradigm performance prompts