MemDPT: Differential Privacy for Memory Efficient Language Models

arXiv:2406.11087v1 Announce Type: new
Abstract: Large language models have consistently demonstrated remarkable performance across a wide spectrum of applications. Nonetheless, the deployment of these models can inadvertently expose user privacy to potential risks. The substantial memory demands of these models during training represent a significant resource consumption challenge. The sheer size of these models imposes a considerable burden on memory resources, which is a matter of significant concern in practice. In this paper, we present an innovative training framework MemDPT …

applications arxiv can challenge cs.ai cs.cl cs.cr cs.lg demands deployment differential privacy expose language language models large memory performance privacy resource risks size spectrum training user privacy