Chaos of Learning Beyond Zero-sum and Coordination via Game Decompositions

It is of primary interest for AI/ML to understand how agents learn and interact dynamically in competitive environments and games (e.g. GANs). But over the past few decades, this has been shown to be a difficult task, as irregular behaviors are commonly observed in such systems. This can be explained theoretically, for instance, by the works of Cheung and Piliouras (COLT 2019, NeurIPS 2020), which showed that in two-person zero-sum games, if agents employ one of the most well-known learning algorithms, Multiplicative Weights Update (MWU), then Lyapunov chaos occurs everywhere in the payoff space. Understanding chaos is an important but long-ignored aspect of AI/ML, as chaos captures the sensitivity to round-off errors, which exist in every step of learning processes. Thus, chaos can severely affect predictability and reproducibility of learning systems. We study how persistent chaos can occur in the more general normal game settings, where the agents might have motivation to coordinate (which is not true for zero-sum games) and the number of agents can be arbitrary. We characterize bimatrix games where MWU, its optimistic variant (OMWU) or Follow-the-Regularized-Leader (FTRL) algorithms is Lyapunov chaotic almost everywhere in the payoff space. Technically, our characterization is derived by extending the volume-expansion argument of Cheung and Piliouras via the canonical game decomposition into zero-sum and coordination components. Interestingly, the two components induce opposite volume-changing behaviors, so the overall behavior can be analysed by comparing the strengths of the components against each other. The comparison is done via our new notion of "matrix domination" or via a linear program. For multi-player games, we present a local equivalence of volume change between general games and graphical games, which is used to perform volume and chaos analyses of MWU and OMWU in potential games.