Reducing Exploitability with Population Based Training. (arXiv:2208.05083v2 [cs.LG] UPDATED)

Self-play reinforcement learning has achieved state-of-the-art, and often
superhuman, performance in a variety of zero-sum games. Yet prior work has
found that policies that are highly capable against regular opponents can fail
catastrophically against adversarial policies: an opponent trained explicitly
against the victim. Prior defenses using adversarial training were able to make
the victim robust to a specific adversary, but the victim remained vulnerable
to new ones. We conjecture this limitation was due to insufficient diversity of
adversaries seen during …

training