The DQN Replay Dataset was collected as follows: We first train a DQN agent, on all 60 Atari 2600 games with sticky actions enabled for 200 million frames (standard protocol) and save all of the experience tuples of (observation, action, reward, next observation) (approximately 50 million) encountered during training.
After installing gsutil, run the command to copy the entire dataset:
gsutil -m cp -R gs://atari-replay-datasets/dqn
To run the dataset only for a specific Atari 2600 game (e.g., replace
Pong to download the logged DQN replay datasets for the game of Pong),
run the command:
gsutil -m cp -R gs://atari-replay-datasets/dqn/[GAME_NAME]
This data can be generated by running the online agents using
batch_rl/baselines/train.py for 200 million frames
(standard protocol). Note that the dataset consists of approximately 50 million
experience tuples due to frame skipping (i.e., repeating a selected action for
k consecutive frames) of 4. The stickiness parameter is set to 0.25, i.e.,
there is 25% chance at every time step that the environment will execute the
agent's previous action again, instead of the agent's new action.