Jailbreaking is Best Solved by Definition

arXiv:2403.14725v1 Announce Type: new
Abstract: The rise of "jailbreak" attacks on language models has led to a flurry of defenses aimed at preventing the output of undesirable responses. In this work, we critically examine the two stages of the defense pipeline: (i) the definition of what constitutes unsafe outputs, and (ii) the enforcement of the definition via methods such as input processing or fine-tuning. We cast severe doubt on the efficacy of existing enforcement mechanisms by showing that they fail …

arxiv attacks cs.cl cs.cr cs.lg defense defenses definition enforcement jailbreak jailbreaking language language models led pipeline work