DP-RAFT: A Differentially Private Recipe for Accelerated Fine-Tuning. (arXiv:2212.04486v1 [cs.LG])

A major direction in differentially private machine learning is
differentially private fine-tuning: pretraining a model on a source of "public
data" and transferring the extracted features to downstream tasks.

This is an important setting because many industry deployments fine-tune
publicly available feature extractors on proprietary data for downstream tasks.

In this paper, we use features extracted from state-of-the-art open source
models to solve benchmark tasks in computer vision and natural language
processing using differentially private fine-tuning. Our key insight is …