Investigating the prompt leakage effect and black-box defenses for multi-turn LLM interactions

arXiv:2404.16251v1 Announce Type: new
Abstract: Prompt leakage in large language models (LLMs) poses a significant security and privacy threat, particularly in retrieval-augmented generation (RAG) systems. However, leakage in multi-turn LLM interactions along with mitigation strategies has not been studied in a standardized manner. This paper investigates LLM vulnerabilities against prompt leakage across 4 diverse domains and 10 closed- and open-source LLMs. Our unique multi-turn threat model leverages the LLM's sycophancy effect and our analysis dissects task instruction and knowledge leakage …

arxiv box cs.ai cs.cl cs.cr defenses effect language language models large llm llms mitigation mitigation strategies privacy prompt rag security strategies systems threat turn vulnerabilities