Goal-guided Generative Prompt Injection Attack on Large Language Models

arXiv:2404.07234v1 Announce Type: new
Abstract: Current large language models (LLMs) provide a strong foundation for large-scale user-oriented natural language tasks. A large number of users can easily inject adversarial text or instructions through the user interface, thus causing LLMs model security challenges. Although there is currently a large amount of research on prompt injection attacks, most of these black-box attacks use heuristic strategies. It is unclear how these heuristic strategies relate to the success rate of attacks and thus effectively …

adversarial arxiv attack can challenges cs.ai cs.cl cs.cr current foundation generative goal inject injection injection attack instructions interface language language models large llms natural natural language prompt prompt injection research scale security security challenges text user interface