all InfoSec news
Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems
Feb. 29, 2024, 5:11 a.m. | Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, Himabindu Lakkaraju
cs.CR updates on arXiv.org arxiv.org
Abstract: Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation. We study the risk of datastore leakage in Retrieval-In-Context RAG Language Models (LMs). We show that an adversary can exploit LMs' instruction-following capabilities to easily extract text data verbatim from the datastore of RAG systems built with instruction-tuned LMs via prompt injection. The vulnerability exists for a wide range of modern LMs that span Llama2, Mistral/Mixtral, Vicuna, SOLAR, …
adaptation adversary arxiv can capabilities context cs.ai cs.cl cs.cr cs.lg data enable exploit external extraction knowledge language language models lms rag risk study systems test
More from arxiv.org / cs.CR updates on arXiv.org
Jobs in InfoSec / Cybersecurity
XDR Detection Engineer
@ SentinelOne | Italy
Security Engineer L2
@ NTT DATA | A Coruña, Spain
Cyber Security Assurance Manager
@ Babcock | Portsmouth, GB, PO6 3EN
Senior Threat Intelligence Researcher
@ CloudSEK | Bengaluru, Karnataka, India
Cybersecurity Analyst 1
@ Spry Methods | Washington, DC (Hybrid)
Security Infrastructure DevOps Engineering Manager
@ Apple | Austin, Texas, United States