Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach

arXiv:2311.14632v2 Announce Type: replace-cross
Abstract: Differentially Private Stochastic Gradient Descent with Gradient Clipping (DPSGD-GC) is a powerful tool for training deep learning models using sensitive data, providing both a solid theoretical privacy guarantee and high efficiency. However, using DPSGD-GC to ensure Differential Privacy (DP) comes at the cost of model performance degradation due to DP noise injection and gradient clipping. Existing research has extensively analyzed the theoretical convergence of DPSGD-GC, and has shown that it only converges when using large …

arxiv bias cost cs.cr cs.lg data deep learning differential privacy efficiency error feedback guarantee high performance privacy private sensitive sensitive data solid tool training