Talk Too Much: Poisoning Large Language Models under Token Limit

arXiv:2404.14795v1 Announce Type: cross
Abstract: Mainstream poisoning attacks on large language models (LLMs) typically set a fixed trigger in the input instance and specific responses for triggered queries. However, the fixed trigger setting (e.g., unusual words) may be easily detected by human detection, limiting the effectiveness and practicality in real-world scenarios. To enhance the stealthiness of the trigger, we present a poisoning attack against LLMs that is triggered by a generation/output condition-token limitation, which is a commonly adopted strategy by …

arxiv attacks cs.cl cs.cr cs.lg detection human input instance language language models large limit llms mainstream may poisoning poisoning attacks real token trigger under world