Agent57: Outperforming the Atari Human Benchmark

Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing outstandingly well on many games of the set, but very poorly in several of the most challenging games. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games. To achieve this result, we train a neural network which parameterizes a family of policies ranging from very exploratory to purely exploitative. We propose an adaptive mechanism to choose which policy to prioritize throughout the training process. Additionally, we utilize a novel parameterization of the architecture that allows for more consistent and stable learning.

PDF Abstract ICML 2020 PDF

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Atari Games Atari 2600 Alien Agent57 Score 297638.17 # 2
Atari Games Atari 2600 Amidar Agent57 Score 29660.08 # 1
Atari Games Atari 2600 Assault Agent57 Score 67212.67 # 4
Atari Games Atari 2600 Asterix Agent57 Score 991384.42 # 4
Atari Games Atari 2600 Asteroids Agent57 Score 150854.61 # 8
Atari Games Atari 2600 Atlantis Agent57 Score 1528841.76 # 7
Atari Games Atari 2600 Bank Heist Agent57 Score 23071.5 # 3
Atari Games Atari 2600 Battle Zone Agent57 Score 934134.88 # 1
Atari Games Atari 2600 Beam Rider Agent57 Score 300509.8 # 4
Atari Games Atari 2600 Berzerk Agent57 Score 61507.83 # 3
Atari Games Atari 2600 Bowling Agent57 Score 251.18 # 3
Atari Games Atari 2600 Boxing Agent57 Score 100 # 1
Atari Games Atari 2600 Breakout Agent57 Score 790.4 # 10
Atari Games Atari 2600 Centipede Agent57 Score 412847.86 # 6
Atari Games Atari 2600 Chopper Command Agent57 Score 999900 # 4
Atari Games Atari 2600 Crazy Climber Agent57 Score 565909.85 # 1
Atari Games Atari 2600 Defender Agent57 Score 677642.78 # 6
Atari Games Atari 2600 Demon Attack Agent57 Score 143161.44 # 7
Atari Games Atari 2600 Double Dunk Agent57 Score 23.93 # 6
Atari Games Atari 2600 Enduro Agent57 Score 2367.71 # 7
Atari Games Atari 2600 Fishing Derby Agent57 Score 86.97 # 2
Atari Games Atari 2600 Freeway Agent57 Score 32.59 # 24
Atari Games Atari 2600 Frostbite Agent57 Score 541280.88 # 2
Atari Games Atari 2600 Gopher Agent57 Score 117777.08 # 8
Atari Games Atari 2600 Gravitar Agent57 Score 19213.96 # 1
Atari Games Atari 2600 HERO Agent57 Score 114736.26 # 1
Atari Games Atari 2600 Ice Hockey Agent57 Score 63.64 # 4
Atari Games Atari 2600 James Bond Agent57 Score 135784.96 # 4
Atari Games Atari 2600 Kangaroo Agent57 Score 24034.16 # 1
Atari Games Atari 2600 Krull Agent57 Score 251997.31 # 3
Atari Games Atari 2600 Kung-Fu Master Agent57 Score 206845.82 # 4
Atari Games Atari 2600 Montezuma's Revenge Agent57 Score 9352.01 # 4
Atari Games Atari 2600 Ms. Pacman Agent57 Score 63994.44 # 3
Atari Games Atari 2600 Name This Game Agent57 Score 54386.77 # 4
Atari Games Atari 2600 Phoenix Agent57 Score 908264.15 # 3
Atari Games Atari 2600 Pitfall! Agent57 Score 18756.01 # 2
Atari Games Atari 2600 Pong Agent57 Score 20.67 # 23
Atari Games Atari 2600 Private Eye Agent57 Score 79716.46 # 2
Atari Games Atari 2600 Q*Bert Agent57 Score 580328.14 # 1
Atari Games Atari 2600 River Raid Agent57 Score 63318.67 # 4
Atari Games Atari 2600 Road Runner Agent57 Score 243025.8 # 7
Atari Games Atari 2600 Robotank Agent57 Score 127.32 # 2
Atari Games Atari 2600 Seaquest Agent57 Score 999997.63 # 3
Atari Games Atari 2600 Skiing Agent57 Score -4202.6 # 3
Atari Games Atari 2600 Solaris Agent57 Score 44199.93 # 1
Atari Games Atari 2600 Space Invaders Agent57 Score 48680.86 # 7
Atari Games Atari 2600 Star Gunner Agent57 Score 839573.53 # 1
Atari Games Atari 2600 Surround Agent57 Score 9.5 # 5
Atari Games Atari 2600 Tennis Agent57 Score 23.84 # 5
Atari Games Atari 2600 Time Pilot Agent57 Score 405425.31 # 5
Atari Games Atari 2600 Tutankham Agent57 Score 2354.91 # 1
Atari Games Atari 2600 Up and Down Agent57 Score 623805.73 # 7
Atari Games Atari 2600 Venture Agent57 Score 2623.71 # 1
Atari Games Atari 2600 Video Pinball Agent57 Score 992340.74 # 2
Atari Games Atari 2600 Wizard of Wor Agent57 Score 157306.41 # 2
Atari Games Atari 2600 Yars Revenge Agent57 Score 998532.37 # 1
Atari Games Atari 2600 Zaxxon Agent57 Score 249808.9 # 2
Atari Games atari game Agent57 Human World Record Breakthrough 18 # 3
Atari Games Atari games Agent57 Mean Human Normalized Score 4763.69% # 5

Methods


No methods listed for this paper. Add relevant methods here