Defending Multimodal Fusion Models against Single-Source Adversaries. (arXiv:2206.12714v1 [cs.CV])

Beyond achieving high performance across many vision tasks, multimodal models
are expected to be robust to single-source faults due to the availability of
redundant information between modalities. In this paper, we investigate the
robustness of multimodal neural networks against worst-case (i.e., adversarial)
perturbations on a single modality. We first show that standard multimodal
fusion models are vulnerable to single-source adversaries: an attack on any
single modality can overcome the correct information from multiple unperturbed
modalities and cause the model to …

adversaries fusion single