RoBIC: A benchmark suite for assessing classifiers robustness. (arXiv:2102.05368v2 [cs.CV] UPDATED)

Many defenses have emerged with the development of adversarial attacks.
Models must be objectively evaluated accordingly. This paper systematically
tackles this concern by proposing a new parameter-free benchmark we coin RoBIC.
RoBIC fairly evaluates the robustness of image classifiers using a new
half-distortion measure. It gauges the robustness of the network against white
and black box attacks, independently of its accuracy. RoBIC is faster than the
other available benchmarks. We present the significant differences in the
robustness of 16 recent …

adversarial adversarial attacks attacks benchmark benchmarks black box box development free measure network parameter robustness