Count-Based Exploration with Neural Density Models

Bellemare et al. (2016) introduced the notion of a pseudo-count, derived from a density model, to generalize count-based exploration to non-tabular reinforcement learning. This pseudo-count was used to generate an exploration bonus for a DQN agent and combined with a mixed Monte Carlo update was sufficient to achieve state of the art on the Atari 2600 game Montezuma's Revenge... (read more)

PDF Abstract ICML 2017 PDF ICML 2017 Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Atari Games Atari 2600 Freeway DQN-CTS Score 33.0 # 7
Atari Games Atari 2600 Freeway DQN-PixelCNN Score 31.7 # 14
Atari Games Atari 2600 Gravitar DQN-CTS Score 238.0 # 40
Atari Games Atari 2600 Gravitar DQN-PixelCNN Score 498.3 # 22
Atari Games Atari 2600 Montezuma's Revenge DQN-PixelCNN Score 3705.5 # 6
Atari Games Atari 2600 Private Eye DQN-CTS Score 206.0 # 26
Atari Games Atari 2600 Private Eye DQN-PixelCNN Score 8358.7 # 7
Atari Games Atari 2600 Venture DQN-CTS Score 48.0 # 28
Atari Games Atari 2600 Venture DQN-PixelCNN Score 82.2 # 25

Methods used in the Paper