How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

arXiv:2402.11173v1 Announce Type: cross
Abstract: We provide a simple and flexible framework for designing differentially private algorithms to find approximate stationary points of non-convex loss functions. Our framework is based on using a private approximate risk minimizer to "warm start" another private algorithm for finding stationary points. We use this framework to obtain improved, and sometimes optimal, rates for several classes of non-convex loss functions. First, we obtain improved rates for finding stationary points of smooth non-convex empirical loss functions. …

algorithm algorithms arxiv cs.cr cs.lg find framework functions loss math.oc non optimization points private privately risk simple start