The importance of feature preprocessing for differentially private linear optimization

arXiv:2307.11106v2 Announce Type: replace-cross
Abstract: Training machine learning models with differential privacy (DP) has received increasing interest in recent years. One of the most popular algorithms for training differentially private models is differentially private stochastic gradient descent (DPSGD) and its variants, where at each step gradients are clipped and combined with some noise. Given the increasing usage of DPSGD, we ask the question: is DPSGD alone sufficient to find a good minimizer for every dataset under privacy constraints? Towards answering …

algorithms arxiv cs.cr cs.it cs.lg differential privacy feature interest linear machine machine learning machine learning models math.it optimization popular privacy private training