The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning

ICLR 2018 Audrunas GruslysWill DabneyMohammad Gheshlaghi AzarBilal PiotMarc BellemareRemi Munos

In this work we present a new agent architecture, called Reactor, which combines multiple algorithmic and architectural contributions to produce an agent with higher sample-efficiency than Prioritized Dueling DQN (Wang et al., 2016) and Categorical DQN (Bellemare et al., 2017), while giving better run-time performance than A3C (Mnih et al., 2016). Our first contribution is a new policy evaluation algorithm called Distributional Retrace, which brings multi-step off-policy updates to the distributional reinforcement learning setting... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Atari Games Atari 2600 Alien Reactor 500M Score 12689.1 # 6
Atari Games Atari 2600 Amidar Reactor 500M Score 1015.8 # 18
Atari Games Atari 2600 Assault Reactor 500M Score 8323.3 # 12
Atari Games Atari 2600 Asterix Reactor 500M Score 205914.0 # 11
Atari Games Atari 2600 Asteroids Reactor 500M Score 3726.1 # 11
Atari Games Atari 2600 Atlantis Reactor 500M Score 302831.0 # 27
Atari Games Atari 2600 Bank Heist Reactor 500M Score 1259.7 # 8
Atari Games Atari 2600 Battle Zone Reactor 500M Score 64070.0 # 5
Atari Games Atari 2600 Beam Rider Reactor 500M Score 11033.4 # 23
Atari Games Atari 2600 Berzerk Reactor 500M Score 2303.1 # 8
Atari Games Atari 2600 Bowling Reactor 500M Score 81.0 # 10
Atari Games Atari 2600 Boxing Reactor 500M Score 99.4 # 6
Atari Games Atari 2600 Breakout Reactor 500M Score 514.8 # 13
Atari Games Atari 2600 Centipede Reactor 500M Score 3422.0 # 32
Atari Games Atari 2600 Chopper Command Reactor 500M Score 107779.0 # 5
Atari Games Atari 2600 Crazy Climber Reactor 500M Score 236422.0 # 4
Atari Games Atari 2600 Defender Reactor 500M Score 223025.0 # 5
Atari Games Atari 2600 Demon Attack Reactor 500M Score 115154.0 # 10
Atari Games Atari 2600 Double Dunk Reactor 500M Score 23.0 # 4
Atari Games Atari 2600 Enduro Reactor 500M Score 2224.2 # 8
Atari Games Atari-57 Reactor Medium Human-Normalized Score 187.0% # 9

Methods used in the Paper