Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent. (arXiv:2102.05855v5 [stat.ML] UPDATED)

What is the information leakage of an iterative randomized learning algorithm
about its training data, when the internal state of the algorithm is
\emph{private}? How much is the contribution of each specific training epoch to
the information leakage through the released model? We study this problem for
noisy gradient descent algorithms, and model the \emph{dynamics} of R\'enyi
differential privacy loss throughout the training process. Our analysis traces
a provably \emph{tight} bound on the R\'enyi divergence between the pair of
probability …

differential privacy privacy