Distilling Adversarial Robustness Using Heterogeneous Teachers

arXiv:2402.15586v1 Announce Type: cross
Abstract: Achieving resiliency against adversarial attacks is necessary prior to deploying neural network classifiers in domains where misclassification incurs substantial costs, e.g., self-driving cars or medical imaging. Recent work has demonstrated that robustness can be transferred from an adversarially trained teacher to a student model using knowledge distillation. However, current methods perform distillation using a single adversarial and vanilla teacher and consider homogeneous architectures (i.e., residual networks) that are susceptible to misclassify examples from similar adversarial …

adversarial adversarial attacks arxiv attacks can cars cs.cr cs.cv current domains driving imaging knowledge medical medical imaging network neural network resiliency robustness self-driving self-driving cars student teachers work