Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance. (arXiv:2303.01256v1 [stat.ML])

Differentially private stochastic gradient descent privatizes model training
by injecting noise into each iteration, where the noise magnitude increases
with the number of model parameters. Recent works suggest that we can reduce
the noise by leveraging public data for private machine learning, by projecting
gradients onto a subspace prescribed by the public data. However, given a
choice of public datasets, it is not a priori clear which one may be most
appropriate for the private task. We give an algorithm …

data datasets machine machine learning magnitude model training noise private public training