Take a Look at it! Rethinking How to Evaluate Language Model Jailbreak

arXiv:2404.06407v1 Announce Type: cross
Abstract: Large language models (LLMs) have become increasingly integrated with various applications. To ensure that LLMs do not generate unsafe responses, they are aligned with safeguards that specify what content is restricted. However, such alignment can be bypassed to produce prohibited content using a technique commonly referred to as jailbreak. Different systems have been proposed to perform the jailbreak automatically. These systems rely on evaluation methods to determine whether a jailbreak attempt is successful. However, our …

alignment applications arxiv can cs.ai cs.cl cs.cr cs.lg jailbreak language language models large llms restricted safeguards