Off-Policy TD Control


Q-Learning is an off-policy temporal difference control algorithm:

$$Q\left(S_{t}, A_{t}\right) \leftarrow Q\left(S_{t}, A_{t}\right) + \alpha\left[R_{t+1} + \gamma\max_{a}Q\left(S_{t+1}, a\right) - Q\left(S_{t}, A_{t}\right)\right] $$

The learned action-value function $Q$ directly approximates $q_{*}$, the optimal action-value function, independent of the policy being followed.

Source: Sutton and Barto, Reinforcement Learning, 2nd Edition


Paper Code Results Date Stars


Task Papers Share
Multi-agent Reinforcement Learning 41 12.17%
Decision Making 33 9.79%
Atari Games 23 6.82%
Offline RL 17 5.04%
Continuous Control 14 4.15%
OpenAI Gym 13 3.86%
Autonomous Driving 10 2.97%
Meta-Learning 9 2.67%
Starcraft 9 2.67%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign