Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise

no code implementations • 4 Feb 2020 • Maxim Kaledin, Eric Moulines, Alexey Naumov, Vladislav Tadic, Hoi-To Wai

Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise, only the constants are affected by the mixing time of the Markov chain.

Reinforcement Learning (RL)