Prioritized Experience Replay

18 Nov 2015Tom SchaulJohn QuanIoannis AntonoglouDavid Silver

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Atari Games Atari 2600 Alien Prior noop Score 4203.8 # 3
Atari Games Atari 2600 Alien Prior hs Score 1334.7 # 12
Atari Games Atari 2600 Amidar Prior noop Score 1838.9 # 4
Atari Games Atari 2600 Amidar Prior hs Score 129.1 # 20
Atari Games Atari 2600 Assault Prior noop Score 7672.1 # 7
Atari Games Atari 2600 Assault Prior hs Score 6548.9 # 9
Atari Games Atari 2600 Asterix Prior noop Score 31527.0 # 5
Atari Games Atari 2600 Asterix Prior hs Score 22484.5 # 7
Atari Games Atari 2600 Asteroids Prior noop Score 2654.3 # 7
Atari Games Atari 2600 Asteroids Prior hs Score 1745.1 # 9
Atari Games Atari 2600 Atlantis Prior noop Score 357324.0 # 13
Atari Games Atari 2600 Atlantis Prior hs Score 330647.0 # 15
Atari Games Atari 2600 Bank Heist Prior noop Score 1054.6 # 7
Atari Games Atari 2600 Bank Heist Prior hs Score 876.6 # 15
Atari Games Atari 2600 Battle Zone Prior noop Score 31530.0 # 6
Atari Games Atari 2600 Battle Zone Prior hs Score 25520.0 # 12
Atari Games Atari 2600 Beam Rider Prior hs Score 31181.3 # 3
Atari Games Atari 2600 Beam Rider Prior noop Score 23384.2 # 7
Atari Games Atari 2600 Berzerk Prior noop Score 1305.6 # 6
Atari Games Atari 2600 Berzerk Prior hs Score 865.9 # 12
Atari Games Atari 2600 Bowling Prior hs Score 52.0 # 11
Atari Games Atari 2600 Bowling Prior noop Score 47.9 # 13
Atari Games Atari 2600 Boxing Prior noop Score 95.6 # 6
Atari Games Atari 2600 Boxing Prior hs Score 72.3 # 14
Atari Games Atari 2600 Breakout Prior noop Score 373.9 # 11
Atari Games Atari 2600 Breakout Prior hs Score 343.0 # 18
Atari Games Atari 2600 Centipede Prior noop Score 4463.2 # 16
Atari Games Atari 2600 Centipede Prior hs Score 3489.1 # 20
Atari Games Atari 2600 Chopper Command Prior noop Score 8600.0 # 6
Atari Games Atari 2600 Chopper Command Prior hs Score 4635.0 # 14
Atari Games Atari 2600 Crazy Climber Prior noop Score 141161.0 # 5
Atari Games Atari 2600 Crazy Climber Prior hs Score 127512.0 # 9
Atari Games Atari 2600 Demon Attack Prior noop Score 71846.4 # 9
Atari Games Atari 2600 Demon Attack Prior hs Score 61277.5 # 12
Atari Games Atari 2600 Double Dunk Prior noop Score 18.5 # 1
Atari Games Atari 2600 Double Dunk Prior hs Score 16.0 # 2
Atari Games Atari 2600 Enduro Prior noop Score 2093.0 # 6
Atari Games Atari 2600 Enduro Prior hs Score 1831.0 # 9
Atari Games Atari 2600 Fishing Derby Prior noop Score 39.5 # 4
Atari Games Atari 2600 Fishing Derby Prior hs Score 9.8 # 12
Atari Games Atari 2600 Freeway Prior noop Score 33.7 # 3
Atari Games Atari 2600 Freeway Prior hs Score 28.9 # 13
Atari Games Atari 2600 Frostbite Prior noop Score 4380.1 # 4
Atari Games Atari 2600 Frostbite Prior hs Score 3510.0 # 8
Atari Games Atari 2600 Gopher Prior hs Score 34858.8 # 5
Atari Games Atari 2600 Gopher Prior noop Score 32487.2 # 7
Atari Games Atari 2600 Gravitar Prior noop Score 548.5 # 5
Atari Games Atari 2600 Gravitar Prior hs Score 269.5 # 20
Atari Games Atari 2600 HERO Prior noop Score 23037.7 # 6
Atari Games Atari 2600 HERO Prior hs Score 20889.9 # 9
Atari Games Atari 2600 Ice Hockey Prior noop Score 1.3 # 1
Atari Games Atari 2600 Ice Hockey Prior hs Score -0.2 # 4
Atari Games Atari 2600 James Bond Prior noop Score 5148.0 # 2
Atari Games Atari 2600 James Bond Prior hs Score 3961.0 # 3
Atari Games Atari 2600 Kangaroo Prior noop Score 16200.0 # 1
Atari Games Atari 2600 Kangaroo Prior hs Score 12185.0 # 8
Atari Games Atari 2600 Krull Prior noop Score 9728.0 # 6
Atari Games Atari 2600 Krull Prior hs Score 6872.8 # 14
Atari Games Atari 2600 Kung-Fu Master Prior noop Score 39581.0 # 5
Atari Games Atari 2600 Kung-Fu Master Prior hs Score 31676.0 # 10
Atari Games Atari 2600 Montezuma's Revenge Prior hs Score 51.0 # 16
Atari Games Atari 2600 Montezuma's Revenge Prior noop Score 0.0 # 23
Atari Games Atari 2600 Ms. Pacman Prior noop Score 6518.7 # 1
Atari Games Atari 2600 Ms. Pacman Prior hs Score 1865.9 # 12
Atari Games Atari 2600 Name This Game Prior noop Score 12270.5 # 6
Atari Games Atari 2600 Name This Game Prior hs Score 10497.6 # 12
Atari Games Atari 2600 Pong Prior noop Score 20.6 # 3
Atari Games Atari 2600 Pong Prior hs Score 18.9 # 6
Atari Games Atari 2600 Private Eye Prior hs Score 670.7 # 9
Atari Games Atari 2600 Private Eye Prior noop Score 200.0 # 16
Atari Games Atari 2600 Q*Bert Prior noop Score 16256.5 # 6
Atari Games Atari 2600 Q*Bert Prior hs Score 9944.0 # 17
Atari Games Atari 2600 River Raid Prior noop Score 14522.3 # 8
Atari Games Atari 2600 River Raid Prior hs Score 11807.2 # 12
Atari Games Atari 2600 Road Runner Prior noop Score 57608.0 # 6
Atari Games Atari 2600 Road Runner Prior hs Score 52264.0 # 9
Atari Games Atari 2600 Robotank Prior noop Score 62.6 # 6
Atari Games Atari 2600 Robotank Prior hs Score 56.2 # 12
Atari Games Atari 2600 Seaquest Prior noop Score 26357.8 # 5
Atari Games Atari 2600 Seaquest Prior hs Score 25463.7 # 6
Atari Games Atari 2600 Space Invaders Prior hs Score 3912.1 # 9
Atari Games Atari 2600 Space Invaders Prior noop Score 2865.8 # 11
Atari Games Atari 2600 Star Gunner Prior noop Score 63302.0 # 9
Atari Games Atari 2600 Star Gunner Prior hs Score 61582.0 # 10
Atari Games Atari 2600 Tennis Prior noop Score 0.0 # 8
Atari Games Atari 2600 Tennis Prior hs Score -5.3 # 13
Atari Games Atari 2600 Time Pilot Prior noop Score 9197.0 # 5
Atari Games Atari 2600 Time Pilot Prior hs Score 5963.0 # 13
Atari Games Atari 2600 Tutankham Prior noop Score 204.6 # 7
Atari Games Atari 2600 Tutankham Prior hs Score 56.9 # 19
Atari Games Atari 2600 Up and Down Prior noop Score 16154.1 # 14
Atari Games Atari 2600 Up and Down Prior hs Score 12157.4 # 16
Atari Games Atari 2600 Venture Prior hs Score 94.0 # 16
Atari Games Atari 2600 Venture Prior noop Score 54.0 # 19
Atari Games Atari 2600 Video Pinball Prior hs Score 295972.8 # 10
Atari Games Atari 2600 Video Pinball Prior noop Score 282007.3 # 11
Atari Games Atari 2600 Wizard of Wor Prior hs Score 5727.0 # 13
Atari Games Atari 2600 Wizard of Wor Prior noop Score 4802.0 # 15
Atari Games Atari 2600 Zaxxon Prior noop Score 10469.0 # 10
Atari Games Atari 2600 Zaxxon Prior hs Score 9474.0 # 13