The Impact of Task Underspecification in Evaluating Deep Reinforcement Learning

no code implementations • 16 Oct 2022 • Vindula Jayawardana, Catherine Tang, Sirui Li, Dajiang Suo, Cathy Wu

We show that in comparison to evaluating DRL methods on select MDP instances, evaluating the MDP family often yields a substantially different relative ranking of methods, casting doubt on what methods should be considered state-of-the-art.

Decision Making reinforcement-learning +1