Off-Policy TD Control


Q-Learning is an off-policy temporal difference control algorithm:

$$Q\left(S_{t}, A_{t}\right) \leftarrow Q\left(S_{t}, A_{t}\right) + \alpha\left[R_{t+1} + \gamma\max_{a}Q\left(S_{t+1}, a\right) - Q\left(S_{t}, A_{t}\right)\right] $$

The learned action-value function $Q$ directly approximates $q_{*}$, the optimal action-value function, independent of the policy being followed.

Source: Sutton and Barto, Reinforcement Learning, 2nd Edition


Paper Code Results Date Stars


Task Papers Share
Decision Making 30 7.67%
Multi-agent Reinforcement Learning 30 7.67%
Atari Games 24 6.14%
Management 23 5.88%
Offline RL 22 5.63%
Autonomous Driving 14 3.58%
Imitation Learning 11 2.81%
OpenAI Gym 11 2.81%
D4RL 9 2.30%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign