Is the System Message Really Important to Jailbreaks in Large Language Models?

arXiv:2402.14857v1 Announce Type: cross
Abstract: The rapid evolution of Large Language Models (LLMs) has rendered them indispensable in modern society. While security measures are typically in place to align LLMs with human values prior to release, recent studies have unveiled a concerning phenomenon named "jailbreak." This term refers to the unexpected and potentially harmful responses generated by LLMs when prompted with malicious questions. Existing research focuses on generating jailbreak prompts but our study aim to answer a different question: Is …

arxiv cs.ai cs.cl cs.cr human human values important jailbreak language language models large llms message rapid release security security measures society studies system