Improving Adversarial Transferability of Visual-Language Pre-training Models through Collaborative Multimodal Interaction

arXiv:2403.10883v1 Announce Type: cross
Abstract: Despite the substantial advancements in Vision-Language Pre-training (VLP) models, their susceptibility to adversarial attacks poses a significant challenge. Existing work rarely studies the transferability of attacks on VLP models, resulting in a substantial performance gap from white-box attacks. We observe that prior work overlooks the interaction mechanisms between modalities, which plays a crucial role in understanding the intricacies of VLP models. In response, we propose a novel attack, called Collaborative Multimodal Interaction Attack (CMI-Attack), leveraging …

adversarial adversarial attacks arxiv attacks box challenge cs.cr cs.cv cs.mm gap language observe performance studies training work