EW-Tune: A Framework for Privately Fine-Tuning Large Language Models with Differential Privacy. (arXiv:2210.15042v1 [cs.CR])

Pre-trained Large Language Models (LLMs) are an integral part of modern AI
that have led to breakthrough performances in complex AI tasks. Major AI
companies with expensive infrastructures are able to develop and train these
large models with billions and millions of parameters from scratch. Third
parties, researchers, and practitioners are increasingly adopting these
pre-trained models and fine-tuning them on their private data to accomplish
their downstream AI tasks. However, it has been shown that an adversary can
extract/reconstruct the …

differential privacy framework language large privacy