Multiplicative Reweighting for Robust Neural Network Optimization

arXiv:2102.12192v4 Announce Type: replace-cross
Abstract: Neural networks are widespread due to their powerful performance. However, they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with expert advice, where multiplicative weight (MW) updates were recently shown to be robust to moderate data corruptions in expert advice, we propose to use MW for reweighting examples during neural networks optimization. We theoretically establish the convergence of our method when used with gradient descent and prove …

advice arxiv cs.cr cs.lg data expert network networks neural network neural networks noisy optimization performance presence training updates