Studious Bob Fight Back Against Jailbreaking via Prompt Adversarial Tuning

Although Large Language Models (LLMs) have achieved tremendous success in various applications, they are also susceptible to certain prompts that can induce them to bypass built-in safety measures and provide dangerous or illegal content, a phenomenon known as jailbreak. To protect LLMs from producing harmful information, various defense strategies are proposed, with most focusing on content filtering or adversarial training of models. In this paper, we propose an approach named Prompt Adversarial Tuning (PAT) to train a defense control mechanism, …

adversarial applications back bypass can cs.ai cs.cl cs.cr cs.lg defense defense strategies illegal information jailbreak jailbreaking language language models large llms producing prompt prompts protect safety strategies