On the Importance of Architecture and Feature Selection in Differentially Private Machine Learning. (arXiv:2205.06720v1 [cs.CR])

We study a pitfall in the typical workflow for differentially private machine
learning. The use of differentially private learning algorithms in a "drop-in"
fashion -- without accounting for the impact of differential privacy (DP) noise
when choosing what feature engineering operations to use, what features to
select, or what neural network architecture to use -- yields overly complex and
poorly performing models. In other words, by anticipating the impact of DP
noise, a simpler and more accurate alternative model could …

architecture machine machine learning