Defending Against Unforeseen Failure Modes with Latent Adversarial Training

arXiv:2403.05030v1 Announce Type: new
Abstract: AI systems sometimes exhibit harmful unintended behaviors post-deployment. This is often despite extensive diagnostics and debugging by developers. Minimizing risks from models is challenging because the attack surface is so large. It is not tractable to exhaustively search for inputs that may cause a model to fail. Red-teaming and adversarial training (AT) are commonly used to make AI systems more robust. However, they have not been sufficient to avoid many real-world failure modes that differ …

adversarial arxiv attack attack surface cs.ai cs.cr cs.lg debugging defending deployment developers failure inputs large may risks search systems training