(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs. (arXiv:2307.10490v2 [cs.CR] UPDATED)

We demonstrate how images and sounds can be used for indirect prompt and
instruction injection in multi-modal LLMs. An attacker generates an adversarial
perturbation corresponding to the prompt and blends it into an image or audio
recording. When the user asks the (unmodified, benign) model about the
perturbed image or audio, the perturbation steers the model to output the
attacker-chosen text and/or make the subsequent dialog follow the attacker's
instruction. We illustrate this attack with several proof-of-concept examples
targeting LLaVa …

adversarial audio image images injection llms modal recording