Independent Natural Policy Gradient Always Converges in Markov Potential Games

no code implementations • 20 Oct 2021 • Roy Fox, Stephen Mcaleer, Will Overman, Ioannis Panageas

Recent results have shown that independent policy gradient converges in MPGs but it was not known whether Independent Natural Policy Gradient converges in MPGs as well.

Multi-agent Reinforcement Learning

Global Convergence of Multi-Agent Policy Gradient in Markov Potential Games

1 code implementation • NeurIPS 2021 • Stefanos Leonardos, Will Overman, Ioannis Panageas, Georgios Piliouras

Counter-intuitively, insights from normal-form potential games do not carry over as MPGs can consist of settings where state-games can be zero-sum games.