FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Models

arXiv:2404.08631v1 Announce Type: new
Abstract: Few-shot classification with foundation models (e.g., CLIP, DINOv2, PaLM-2) enables users to build an accurate classifier with a few labeled training samples (called support samples) for a classification task. However, an attacker could perform data poisoning attacks by manipulating some support samples such that the classifier makes the attacker-desired, arbitrary prediction for a testing input. Empirical defenses cannot provide formal robustness guarantees, leading to a cat-and-mouse game between the attacker and defender. Existing certified defenses …

arxiv attacker attacks build called classification cs.cr data data poisoning foundation foundation models palm poisoning poisoning attacks support task training