Locally Differentially Private In-Context Learning

arXiv:2405.04032v1 Announce Type: new
Abstract: Large pretrained language models (LLMs) have shown surprising In-Context Learning (ICL) ability. An important application in deploying large language models is to augment LLMs with a private database for some specific task. The main problem with this promising commercial use is that LLMs have been shown to memorize their training data and their prompt data are vulnerable to membership inference attacks (MIA) and prompt leaking attacks. In order to deal with this problem, we treat …

application arxiv commercial context cs.ai cs.cr database important language language models large llms locally main private problem task training