Implicit Quantile Networks for Distributional Reinforcement Learning

In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regression to approximate the full quantile function for the state-action return distribution... (read more)

PDF Abstract ICML 2018 PDF ICML 2018 Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Alien IQN Score 7022 # 9
Atari Games Atari 2600 Amidar IQN Score 2946 # 7
Atari Games Atari 2600 Assault IQN Score 29091 # 4
Atari Games Atari 2600 Asterix IQN Score 342016 # 8
Atari Games Atari 2600 Asteroids IQN Score 2898 # 15
Atari Games Atari 2600 Atlantis IQN Score 978200 # 10
Atari Games Atari 2600 Bank Heist IQN Score 1416 # 6
Atari Games Atari 2600 Battle Zone IQN Score 42244 # 9
Atari Games Atari 2600 Beam Rider IQN Score 42776 # 5
Atari Games Atari 2600 Berzerk IQN Score 1053 # 21
Atari Games Atari 2600 Bowling IQN Score 86.5 # 8
Atari Games Atari 2600 Boxing IQN Score 99.8 # 4
Atari Games Atari 2600 Breakout IQN Score 734 # 11
Atari Games Atari 2600 Centipede IQN Score 11561 # 10
Atari Games Atari 2600 Chopper Command IQN Score 16836 # 9
Atari Games Atari 2600 Crazy Climber IQN Score 179082 # 8
Atari Games Atari 2600 Defender IQN Score 53537 # 8
Atari Games Atari 2600 Demon Attack IQN Score 128580 # 8
Atari Games Atari 2600 Double Dunk IQN Score 5.6 # 12
Atari Games Atari 2600 Enduro IQN Score 2359 # 5
Atari Games Atari 2600 Fishing Derby IQN Score 33.8 # 15
Atari Games Atari 2600 Freeway IQN Score 34 # 1
Atari Games Atari 2600 Frostbite IQN Score 4324 # 13
Atari Games Atari 2600 Gopher IQN Score 118365 # 4
Atari Games Atari 2600 Gravitar IQN Score 911 # 16
Atari Games Atari 2600 HERO IQN Score 28386 # 13
Atari Games Atari 2600 Ice Hockey IQN Score 0.2 # 13
Atari Games Atari 2600 James Bond IQN Score 35108 # 4
Atari Games Atari 2600 Kangaroo IQN Score 15487 # 4
Atari Games Atari 2600 Krull IQN Score 10707 # 9
Atari Games Atari 2600 Kung-Fu Master IQN Score 73512 # 7
Atari Games Atari 2600 Montezuma's Revenge IQN Score 0 # 35
Atari Games Atari 2600 Ms. Pacman IQN Score 6349 # 10
Atari Games Atari 2600 Name This Game IQN Score 22682 # 5
Atari Games Atari 2600 Phoenix IQN Score 56599 # 8
Atari Games Atari 2600 Pitfall! IQN Score 0 # 4
Atari Games Atari 2600 Pong IQN Score 21 # 1
Atari Games Atari 2600 Private Eye IQN Score 200 # 27
Atari Games Atari 2600 Q*Bert IQN Score 25750 # 10
Atari Games Atari 2600 River Raid IQN Score 17765 # 9
Atari Games Atari 2600 Road Runner IQN Score 57900 # 12
Atari Games Atari 2600 Robotank IQN Score 62.5 # 14
Atari Games Atari 2600 Seaquest IQN Score 30140 # 8
Atari Games Atari 2600 Skiing IQN Score -9289 # 7
Atari Games Atari 2600 Solaris IQN Score 8007 # 4
Atari Games Atari 2600 Space Invaders IQN Score 28888 # 7
Atari Games Atari 2600 Star Gunner IQN Score 74677 # 15
Atari Games Atari 2600 Surround IQN Score 9.4 # 5
Atari Games Atari 2600 Tennis IQN Score 23.6 # 3
Atari Games Atari 2600 Time Pilot IQN Score 12236 # 11
Atari Games Atari 2600 Tutankham IQN Score 293 # 6
Atari Games Atari 2600 Up and Down IQN Score 88148 # 10
Atari Games Atari 2600 Venture IQN Score 1318 # 7
Atari Games Atari 2600 Video Pinball IQN Score 698045 # 8
Atari Games Atari 2600 Wizard of Wor IQN Score 31190 # 7
Atari Games Atari 2600 Yars Revenge IQN Score 28379 # 9
Atari Games Atari 2600 Zaxxon IQN Score 21772 # 9

Methods used in the Paper


METHOD TYPE
Convolution
Convolutions
Dense Connections
Feedforward Networks
Q-Learning
Off-Policy TD Control
DQN
Q-Learning Networks