LeftoverLocals: Listening to LLM Responses Through Leaked GPU Local Memory. (arXiv:2401.16603v1 [cs.CR])

This paper describes LeftoverLocals: a vulnerability that allows data
recovery from GPU memory created by another process on Apple, Qualcomm, and AMD
GPUs. LeftoverLocals impacts the security posture of GPU applications, with
particular significance to LLMs and ML models that run on impacted GPUs. By
recovering local memory, an optimized GPU memory region, we built a PoC where
an attacker can listen into another user's interactive LLM session (e.g.,
llama.cpp) across process or container boundaries.

amd apple applications arxiv data data recovery gpu gpus leaked listening llm llms local memory ml models posture process qualcomm recovery run security security posture vulnerability