Few-Shot Adversarial Prompt Learning on Vision-Language Models

arXiv:2403.14774v1 Announce Type: cross
Abstract: The vulnerability of deep neural networks to imperceptible adversarial perturbations has attracted widespread attention. Inspired by the success of vision-language foundation models, previous efforts achieved zero-shot adversarial robustness by aligning adversarial visual features with text supervision. However, in practice, they are still unsatisfactory due to several issues, including heavy adaptation cost, suboptimal text supervision, and uncontrolled natural generalization capacity. In this paper, to address these issues, we propose a few-shot adversarial prompt framework where adapting …

adversarial arxiv attention cs.cl cs.cr cs.cv cs.lg features foundation foundation models language language models networks neural networks practice prompt robustness text vulnerability