Deep Attention Recurrent Q-Network

A deep learning approach to reinforcement learning led to a general learner able to train on visual input to play a variety of arcade games at the human and superhuman levels. Its creators at the Google DeepMind's team called the approach: Deep Q-Network (DQN). We present an extension of DQN by "soft" and "hard" attention mechanisms. Tests of the proposed Deep Attention Recurrent Q-Network (DARQN) algorithm on multiple Atari 2600 games show level of performance superior to that of DQN. Moreover, built-in attention mechanisms allow a direct online monitoring of the training process by highlighting the regions of the game screen the agent is focusing on when making decisions.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Atari Games Atari 2600 Breakout DARQN hard Score 20 # 52
Atari Games Atari 2600 Gopher DARQN soft Score 5356 # 37
Atari Games Atari 2600 Seaquest DARQN soft Score 7263 # 28
Atari Games Atari 2600 Space Invaders DARQN soft Score 650 # 48
Atari Games Atari 2600 Tutankham DARQN soft Score 197 # 25