LLM Jailbreak Attack versus Defense Techniques -- A Comprehensive Study

arXiv:2402.13457v1 Announce Type: new
Abstract: Large Language Models (LLMS) have increasingly become central to generating content with potential societal impacts. Notably, these models have demonstrated capabilities for generating content that could be deemed harmful. To mitigate these risks, researchers have adopted safety training techniques to align model outputs with societal values to curb the generation of malicious content. However, the phenomenon of "jailbreaking", where carefully crafted prompts elicit harmful responses from models, persists as a significant challenge. This research conducts …

arxiv attack capabilities cs.ai cs.cr defense jailbreak language language models large llm llms researchers risks safety study techniques training