FedEval: A Holistic Evaluation Framework for Federated Learning. (arXiv:2011.09655v3 [cs.LG] UPDATED)

Federated Learning (FL) has been widely accepted as the solution for
privacy-preserving machine learning without collecting raw data. While new
technologies proposed in the past few years do evolve the FL area,
unfortunately, the evaluation results presented in these works fall short in
integrity and are hardly comparable because of the inconsistent evaluation
metrics and experimental settings. In this paper, we propose a holistic
evaluation framework for FL called FedEval, and present a benchmarking study on
seven state-of-the-art FL algorithms. …

evaluation federated learning framework