Defending Against Indirect Prompt Injection Attacks With Spotlighting

arXiv:2403.14720v1 Announce Type: new
Abstract: Large Language Models (LLMs), while powerful, are built and trained to process a single text input. In common applications, multiple inputs can be processed by concatenating them together into a single stream of text. However, the LLM is unable to distinguish which sections of prompt belong to various input sources. Indirect prompt injection attacks take advantage of this vulnerability by embedding adversarial instructions into untrusted data being processed alongside user commands. Often, the LLM will …

applications arxiv attacks can cs.cl cs.cr cs.lg defending injection injection attacks input inputs language language models large llm llms process processed prompt prompt injection prompt injection attacks single stream text