Learning values across many orders of magnitude

NeurIPS 2016 Hado van HasseltArthur GuezMatteo HesselVolodymyr MnihDavid Silver

Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Atari Games Atari 2600 Alien DDQN+Pop-Art noop Score 3213.5 # 6
Atari Games Atari 2600 Amidar DDQN+Pop-Art noop Score 782.5 # 9
Atari Games Atari 2600 Assault DDQN+Pop-Art noop Score 9011.6 # 5
Atari Games Atari 2600 Asterix DDQN+Pop-Art noop Score 18919.5 # 10
Atari Games Atari 2600 Asteroids DDQN+Pop-Art noop Score 2869.3 # 5
Atari Games Atari 2600 Atlantis DDQN+Pop-Art noop Score 340076.0 # 14
Atari Games Atari 2600 Bank Heist DDQN+Pop-Art noop Score 1103.3 # 6
Atari Games Atari 2600 Battle Zone DDQN+Pop-Art noop Score 8220.0 # 21
Atari Games Atari 2600 Beam Rider DDQN+Pop-Art noop Score 8299.4 # 17
Atari Games Atari 2600 Berzerk DDQN+Pop-Art noop Score 1199.6 # 8
Atari Games Atari 2600 Bowling DDQN+Pop-Art noop Score 102.1 # 1
Atari Games Atari 2600 Boxing DDQN+Pop-Art noop Score 99.3 # 3
Atari Games Atari 2600 Breakout DDQN+Pop-Art noop Score 344.1 # 17
Atari Games Atari 2600 Centipede DDQN+Pop-Art noop Score 49065.8 # 1
Atari Games Atari 2600 Chopper Command DDQN+Pop-Art noop Score 775.0 # 21
Atari Games Atari 2600 Crazy Climber DDQN+Pop-Art noop Score 119679.0 # 11
Atari Games Atari 2600 Demon Attack DDQN+Pop-Art noop Score 63644.9 # 11
Atari Games Atari 2600 Double Dunk DDQN+Pop-Art noop Score -11.5 # 16
Atari Games Atari 2600 Enduro DDQN+Pop-Art noop Score 2002.1 # 8
Atari Games Atari 2600 Fishing Derby DDQN+Pop-Art noop Score 45.1 # 2
Atari Games Atari 2600 Freeway DDQN+Pop-Art noop Score 33.4 # 4
Atari Games Atari 2600 Frostbite DDQN+Pop-Art noop Score 3469.6 # 9
Atari Games Atari 2600 Gopher DDQN+Pop-Art noop Score 56218.2 # 4
Atari Games Atari 2600 Gravitar DDQN+Pop-Art noop Score 483.5 # 8
Atari Games Atari 2600 HERO DDQN+Pop-Art noop Score 14225.2 # 18
Atari Games Atari 2600 Ice Hockey DDQN+Pop-Art noop Score -4.1 # 15
Atari Games Atari 2600 James Bond DDQN+Pop-Art noop Score 507.5 # 17
Atari Games Atari 2600 Kangaroo DDQN+Pop-Art noop Score 13150.0 # 5
Atari Games Atari 2600 Krull DDQN+Pop-Art noop Score 9745.1 # 4
Atari Games Atari 2600 Kung-Fu Master DDQN+Pop-Art noop Score 34393.0 # 8
Atari Games Atari 2600 Montezuma's Revenge DDQN+Pop-Art noop Score 0.0 # 23
Atari Games Atari 2600 Ms. Pacman DDQN+Pop-Art noop Score 4963.8 # 4
Atari Games Atari 2600 Name This Game DDQN+Pop-Art noop Score 15851.2 # 2
Atari Games Atari 2600 Pong DDQN+Pop-Art noop Score 20.6 # 3
Atari Games Atari 2600 Private Eye DDQN+Pop-Art noop Score 286.7 # 12
Atari Games Atari 2600 Q*Bert DDQN+Pop-Art noop Score 5236.8 # 20
Atari Games Atari 2600 River Raid DDQN+Pop-Art noop Score 12530.8 # 10
Atari Games Atari 2600 Road Runner DDQN+Pop-Art noop Score 47770.0 # 11
Atari Games Atari 2600 Robotank DDQN+Pop-Art noop Score 64.3 # 4
Atari Games Atari 2600 Seaquest DDQN+Pop-Art noop Score 10932.3 # 9
Atari Games Atari 2600 Space Invaders DDQN+Pop-Art noop Score 2589.7 # 13
Atari Games Atari 2600 Star Gunner DDQN+Pop-Art noop Score 589.0 # 21
Atari Games Atari 2600 Tennis DDQN+Pop-Art noop Score 12.1 # 4
Atari Games Atari 2600 Time Pilot DDQN+Pop-Art noop Score 4870.0 # 18
Atari Games Atari 2600 Tutankham DDQN+Pop-Art noop Score 183.9 # 9
Atari Games Atari 2600 Up and Down DDQN+Pop-Art noop Score 22474.4 # 12
Atari Games Atari 2600 Venture DDQN+Pop-Art noop Score 1172.0 # 4
Atari Games Atari 2600 Video Pinball DDQN+Pop-Art noop Score 56287.0 # 18
Atari Games Atari 2600 Wizard of Wor DDQN+Pop-Art noop Score 483.0 # 21
Atari Games Atari 2600 Zaxxon DDQN+Pop-Art noop Score 14402.0 # 4