Replay Memory

Experience Replay

Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e_{t} = \left(s_{t}, a_{t}, r_{t}, s_{t+1}\right)$ in a data-set $D = e_{1}, \cdots, e_{N}$ , pooled over many episodes into a replay memory. We then usually sample the memory randomly for a minibatch of experience, and use this to learn off-policy, as with Deep Q-Networks. This tackles the problem of autocorrelation leading to unstable training, by making the problem more like a supervised learning problem.

Image Credit: Hands-On Reinforcement Learning with Python, Sudharsan Ravichandiran


Paper Code Results Date Stars


Task Papers Share
reinforcement-learning 296 39.05%
Continuous Control 68 8.97%
Continual Learning 43 5.67%
OpenAI Gym 34 4.49%
Atari Games 27 3.56%
Multi-agent Reinforcement Learning 22 2.90%
Decision Making 21 2.77%
Imitation Learning 17 2.24%
DeepMind 13 1.72%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign