Massively Parallel Methods for Deep Reinforcement Learning

We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Alien Gorila Score 813.5 # 36
Atari Games Atari 2600 Amidar Gorila Score 189.2 # 31
Atari Games Atari 2600 Assault Gorila Score 1195.8 # 34
Atari Games Atari 2600 Asterix Gorila Score 3324.7 # 34
Atari Games Atari 2600 Asteroids Gorila Score 933.6 # 34
Atari Games Atari 2600 Atlantis Gorila Score 629166.5 # 19
Atari Games Atari 2600 Bank Heist Gorila Score 399.4 # 32
Atari Games Atari 2600 Battle Zone Gorila Score 19938.0 # 31
Atari Games Atari 2600 Beam Rider Gorila Score 3822.1 # 36
Atari Games Atari 2600 Bowling Gorila Score 54 # 22
Atari Games Atari 2600 Boxing Gorila Score 74.2 # 22
Atari Games Atari 2600 Breakout Gorila Score 313.0 # 34
Atari Games Atari 2600 Centipede Gorila Score 6296.9 # 20
Atari Games Atari 2600 Chopper Command Gorila Score 3191.8 # 34
Atari Games Atari 2600 Crazy Climber Gorila Score 65451.0 # 34
Atari Games Atari 2600 Demon Attack Gorila Score 14880.1 # 29
Atari Games Atari 2600 Double Dunk Gorila Score -11.3 # 30
Atari Games Atari 2600 Enduro Gorila Score 71.0 # 34
Atari Games Atari 2600 Fishing Derby Gorila Score 4.6 # 27
Atari Games Atari 2600 Freeway Gorila Score 10.2 # 32
Atari Games Atari 2600 Frostbite Gorila Score 426.6 # 34
Atari Games Atari 2600 Gopher Gorila Score 4373.0 # 33
Atari Games Atari 2600 Gravitar Gorila Score 538.4 # 21
Atari Games Atari 2600 HERO Gorila Score 8963.4 # 33
Atari Games Atari 2600 Ice Hockey Gorila Score -1.7 # 20
Atari Games Atari 2600 James Bond Gorila Score 444.0 # 29
Atari Games Atari 2600 Kangaroo Gorila Score 1431.0 # 29
Atari Games Atari 2600 Krull Gorila Score 6363.1 # 30
Atari Games Atari 2600 Kung-Fu Master Gorila Score 20620.0 # 34
Atari Games Atari 2600 Montezuma's Revenge Gorila Score 84 # 21
Atari Games Atari 2600 Ms. Pacman Gorila Score 1263.0 # 32
Atari Games Atari 2600 Name This Game Gorila Score 9238.5 # 26
Atari Games Atari 2600 Pong Gorila Score 16.7 # 17
Atari Games Atari 2600 Private Eye Gorila Score 2598.6 # 11
Atari Games Atari 2600 Q*Bert Gorila Score 7089.8 # 33
Atari Games Atari 2600 River Raid Gorila Score 5310.3 # 29
Atari Games Atari 2600 Road Runner Gorila Score 43079.8 # 25
Atari Games Atari 2600 Robotank Gorila Score 61.8 # 16
Atari Games Atari 2600 Seaquest Gorila Score 10145.9 # 16
Atari Games Atari 2600 Space Invaders Gorila Score 1183.3 # 34
Atari Games Atari 2600 Star Gunner Gorila Score 14919.2 # 31
Atari Games Atari 2600 Tennis Gorila Score -0.7 # 15
Atari Games Atari 2600 Time Pilot Gorila Score 8267.8 # 21
Atari Games Atari 2600 Tutankham Gorila Score 118.5 # 26
Atari Games Atari 2600 Up and Down Gorila Score 8747.7 # 32
Atari Games Atari 2600 Venture Gorila Score 523.4 # 13
Atari Games Atari 2600 Video Pinball Gorila Score 112093.4 # 25
Atari Games Atari 2600 Wizard of Wor Gorila Score 10431.0 # 14
Atari Games Atari 2600 Zaxxon Gorila Score 6159.4 # 29

Methods used in the Paper