Differentially Private Next-Token Prediction of Large Language Models

arXiv:2403.15638v1 Announce Type: new
Abstract: Ensuring the privacy of Large Language Models (LLMs) is becoming increasingly important. The most widely adopted technique to accomplish this is DP-SGD, which trains a model in such a way that guarantees Differential Privacy (DP). However, DP-SGD requires longer training times and larger memory requirements than SGD, while overestimating an adversary's capabilities in having white box access to the model. A more realistic scenario assumes only black-box access to a privacy-sensitive LLM. Motivated by these …

arxiv cs.cl cs.cr cs.lg differential privacy important language language models large llms memory next prediction privacy private requirements token training trains