Off-Policy TD Control


Q-Learning is an off-policy temporal difference control algorithm:

$$Q\left(S_{t}, A_{t}\right) \leftarrow Q\left(S_{t}, A_{t}\right) + \alpha\left[R_{t+1} + \gamma\max_{a}Q\left(S_{t+1}, a\right) - Q\left(S_{t}, A_{t}\right)\right] $$

The learned action-value function $Q$ directly approximates $q_{*}$, the optimal action-value function, independent of the policy being followed.

Source: Sutton and Barto, Reinforcement Learning, 2nd Edition


Paper Code Results Date Stars


Task Papers Share
Atari Games 38 11.45%
Decision Making 38 11.45%
Multi-agent Reinforcement Learning 36 10.84%
OpenAI Gym 19 5.72%
Continuous Control 17 5.12%
Offline RL 12 3.61%
Imitation Learning 7 2.11%
Autonomous Vehicles 7 2.11%
Autonomous Driving 7 2.11%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign