Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger. (arXiv:2206.07136v1 [cs.LG])

Per-example gradient clipping is a key algorithmic step that enables
practical differential private (DP) training for deep learning models. The
choice of clipping norm $R$, however, is shown to be vital for achieving high
accuracy under DP. We propose an easy-to-use replacement, called AutoClipping,
that eliminates the need to tune $R$ for any DP optimizers, including DP-SGD,
DP-Adam, DP-LAMB and many others. The automatic variants are as private and
computationally efficient as existing DP optimizers, but require no DP-specific
hyperparameters …

deep learning lg