Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Losses. (arXiv:2305.09903v1 [cs.LG])

The Noisy-SGD algorithm is widely used for privately training machine
learning models. Traditional privacy analyses of this algorithm assume that the
internal state is publicly revealed, resulting in privacy loss bounds that
increase indefinitely with the number of iterations. However, recent findings
have shown that if the internal state remains hidden, then the privacy loss
might remain bounded. Nevertheless, this remarkable result heavily relies on
the assumption of (strong) convexity of the loss function. It remains an
important open problem …

algorithm converge findings internal loss losses machine machine learning machine learning models non privacy state training