Differentially Private Non-Convex Optimization under the KL Condition with Optimal Rates. (arXiv:2311.13447v1 [cs.LG])

We study private empirical risk minimization (ERM) problem for losses
satisfying the $(\gamma,\kappa)$-Kurdyka-{\L}ojasiewicz (KL) condition. The
Polyak-{\L}ojasiewicz (PL) condition is a special case of this condition when
$\kappa=2$. Specifically, we study this problem under the constraint of $\rho$
zero-concentrated differential privacy (zCDP). When $\kappa\in[1,2]$ and the
loss function is Lipschitz and smooth over a sufficiently large region, we
provide a new algorithm based on variance reduced gradient descent that
achieves the rate
$\tilde{O}\big(\big(\frac{\sqrt{d}}{n\sqrt{\rho}}\big)^\kappa\big)$ on the
excess empirical risk, where $n$ …

case differential privacy loss losses minimization non optimization privacy private problem risk special study under