Variance Reduction for Policy-Gradient Methods via Empirical Variance Minimization

no code implementations • 14 Jun 2022 • Maxim Kaledin, Alexander Golubev, Denis Belomestny

Policy-gradient methods in Reinforcement Learning(RL) are very universal and widely applied in practice but their performance suffers from the high variance of the gradient estimate.

Policy Gradient Methods Reinforcement Learning (RL)

Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise

no code implementations • 4 Feb 2020 • Maxim Kaledin, Eric Moulines, Alexey Naumov, Vladislav Tadic, Hoi-To Wai

Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise, only the constants are affected by the mixing time of the Markov chain.

Reinforcement Learning (RL)