Jatmo: Prompt Injection Defense by Task-Specific Finetuning. (arXiv:2312.17673v1 [cs.CR])

Large Language Models (LLMs) are attracting significant research attention
due to their instruction-following abilities, allowing users and developers to
leverage LLMs for a variety of tasks. However, LLMs are vulnerable to
prompt-injection attacks: a class of attacks that hijack the model's
instruction-following abilities, changing responses to prompts to undesired,
possibly malicious ones. In this work, we introduce Jatmo, a method for
generating task-specific models resilient to prompt-injection attacks. Jatmo
leverages the fact that LLMs can only follow instructions once they …

attacks attention changing class defense developers finetuning hijack injection injection attacks language language models large llms malicious prompt prompt injection prompts research task vulnerable