Exploration is Harder than Prediction: Cryptographically Separating Reinforcement Learning from Supervised Learning

arXiv:2404.03774v1 Announce Type: cross
Abstract: Supervised learning is often computationally easy in practice. But to what extent does this mean that other modes of learning, such as reinforcement learning (RL), ought to be computationally easy by extension? In this work we show the first cryptographic separation between RL and supervised learning, by exhibiting a class of block MDPs and associated decoding functions where reward-free exploration is provably computationally harder than the associated regression problem. We also show that there is …

arxiv cryptographic cs.cc cs.cr cs.ds cs.lg easy exploration extension practice prediction work