Straggler-Resilient Differentially-Private Decentralized Learning

arXiv:2212.03080v3 Announce Type: replace-cross
Abstract: We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that …

amplification analytical arxiv communication computation convergence cs.cr cs.it cs.lg data data privacy decentralization decentralized differential privacy framework latency math.it privacy private problem resilient results ring training user data