Are aligned neural networks adversarially aligned?

arXiv:2306.15447v2 Announce Type: replace-cross
Abstract: Large language models are now tuned to align with the goals of their creators, namely to be "helpful and harmless." These models should respond helpfully to user questions, but refuse to answer requests that could cause harm. However, adversarial users can construct inputs which circumvent attempts at alignment. In this work, we study adversarial alignment, and ask to what extent these models remain aligned when interacting with an adversarial user who constructs worst-case inputs (adversarial …

adversarial align alignment arxiv can creators cs.ai cs.cl cs.cr cs.lg goals harm inputs language language models large networks neural networks questions requests respond