Search Results for author:
Found 62 papers, 14 papers with code
From Chaos to Order: Symmetry and Conservation Laws in Game Dynamics
Even simple games learning dynamics can yield chaotic behavior.
Convex Markov Games: A Framework for Fairness, Imitation, and Creativity in Multi-Agent Learning
Expert imitation, behavioral diversity, and fairness preferences give rise to preferences in sequential decision making domains that do not decompose additively across time.
Swim till You Sink: Computing the Limit of a Game
During 2023, two interesting results were proven about the limit behavior of game dynamics: First, it was shown that there is a game for which no dynamics converges to the Nash equilibria.
Steering game dynamics towards desired outcomes
To tackle this challenge, in this work we introduce the SIAR-MPC method, combining the recently introduced Side Information Assisted Regression (SIAR) method for system identification with Model Predictive Control (MPC).
States as Strings as Strategies: Steering Language Models with Game-Theoretic Solvers
A suitable model of the players, strategies, and payoffs associated with linguistic interactions (i. e., a binding to the conventional symbolic logic of game theory) would enable existing game-theoretic algorithms to provide strategic solutions in the space of language.
Neural Population Learning beyond Symmetric Zero-sum Games
We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game.
Exploiting hidden structures in non-convex games for convergence to Nash equilibrium
A wide array of modern machine learning applications - from adversarial models to multi-agent reinforcement learning - can be formulated as non-cooperative games whose Nash equilibria represent the system's desired operational states.
Scalable AI Safety via Doubly-Efficient Debate
The emergence of pre-trained AI systems with powerful capabilities across a diverse and ever-increasing set of complex domains has raised a critical challenge for AI safety as tasks can become too complicated for humans to judge directly.
A Quadratic Speedup in Finding Nash Equilibria of Quantum Zero-Sum Games
In 2008, Jain and Watrous proposed the first classical algorithm for computing equilibria in quantum zero-sum games using the Matrix Multiplicative Weight Updates (MMWU) method to achieve a convergence rate of $\mathcal{O}(d/\epsilon^2)$ iterations to $\epsilon$-Nash equilibria in the $4^d$-dimensional spectraplex.
Stability of Multi-Agent Learning: Convergence in Network Games with Many Players
The behaviour of multi-agent learning in many player games has been shown to display complex dynamics outside of restrictive examples such as network zero-sum games.
Data-Scarce Identification of Game Dynamics via Sum-of-Squares Optimization
Understanding how players adjust their strategies in games, based on their experience, is a crucial tool for policymakers.
Multiplicative Updates for Online Convex Optimization over Symmetric Cones
We study online convex optimization where the possible actions are trace-one elements in a symmetric cone, generalizing the extensively-studied experts setup and its quantum counterpart.
Chaos persists in large-scale multi-agent learning despite adaptive learning rates
Although such techniques are known to allow for improved convergence guarantees in small games, it has been much harder to analyze them in more relevant settings with large populations of agents.
Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamics
Achieving convergence of multiple learning agents in general $N$-player games is imperative for the development of safe and reliable machine learning (ML) algorithms and their application to autonomous systems.
Min-Max Optimization Made Simple: Approximating the Proximal Point Method via Contraction Maps
In this paper we present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems while requiring a simple and intuitive analysis.
Learning Correlated Equilibria in Mean-Field Games
The designs of many large-scale systems today, from traffic routing environments to smart grids, rely on game-theoretic equilibrium concepts.
Fast Convergence of Optimistic Gradient Ascent in Network Zero-Sum Extensive Form Games
The study of learning in games has thus far focused primarily on normal form games.
Alternating Mirror Descent for Constrained Min-Max Games
In this paper we study two-player bilinear zero-sum games with constrained strategy spaces.
Nash, Conley, and Computation: Impossibility and Incompleteness in Game Dynamics
We also prove a stronger result for $\epsilon$-approximate Nash equilibria: there are games such that no game dynamics can converge (in an appropriate sense) to $\epsilon$-Nash equilibria, and in fact the set of such games has positive measure.
Scalable Deep Reinforcement Learning Algorithms for Mean Field Games
One limiting factor to further scale up using RL is that existing algorithms to solve MFGs require the mixing of approximated quantities such as strategies or $q$-values.
No-Regret Learning in Games is Turing Complete
Games are natural models for multi-agent machine learning settings, such as generative adversarial networks (GANs).
Multi-agent Performative Prediction: From Global Stability and Optimality to Chaos
The recent framework of performative prediction is aimed at capturing settings where predictions influence the target/outcome they want to predict.
Beyond Time-Average Convergence: Near-Optimal Uncoupled Online Learning via Clairvoyant Multiplicative Weights Update
This implies that the CMWU dynamics converge with rate $O(nV \log m \log T / T)$ to a \textit{Coarse Correlated Equilibrium}.
Online Learning in Periodic Zero-Sum Games
Classical learning results build on this theorem to show that online no-regret dynamics converge to an equilibrium in a time-average sense in zero-sum games.
Generalized Natural Gradient Flows in Hidden Convex-Concave Games and GANs
In the case of Fisher information geometry, we provide a complete picture of the dynamics in an interesting special setting of team competition via invariant function analysis.
The Evolution of Uncertainty of Learning in Games
We study standard online learning in games but from a non-standard perspective.
Constants of Motion: The Antidote to Chaos in Optimization and Game Dynamics
Several recent works in online optimization and game dynamics have established strong negative complexity results including the formal emergence of instability and chaos even in small such settings, e. g., $2\times 2$ games.
Evolutionary Dynamics and $Φ$-Regret Minimization in Games
Importantly, $\Phi$-regret enables learning agents to consider deviations from and to mixed strategies, generalizing several existing notions of regret such as external, internal, and swap regret, and thus broadening the insights gained from regret-based analysis of learning algorithms.
Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationality
The interplay between exploration and exploitation in competitive multi-agent learning is still far from being well understood.
Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincaré Recurrence
Passivity is a fundamental concept in control theory, which abstracts energy conservation and dissipation in physical systems.
Efficient Online Learning for Dynamic k-Clustering
We study dynamic clustering problems from the perspective of online learning.
Global Convergence of Multi-Agent Policy Gradient in Markov Potential Games
Counter-intuitively, insights from normal-form potential games do not carry over as MPGs can consist of settings where state-games can be zero-sum games.
Learning in Matrix Games can be Arbitrarily Complex
In this paper we show that, in a strong sense, this dynamic complexity is inherent to games.
Scaling up Mean Field Games with Online Mirror Descent
We address scaling up equilibrium computation in Mean Field Games (MFGs) using Online Mirror Descent (OMD).
Follow-the-Regularized-Leader Routes to Chaos in Routing Games
We establish that, even in simple linear non-atomic congestion games with two parallel links and any fixed learning rate, unless the game is fully symmetric, increasing the population size or the scale of costs causes learning dynamics to become unstable and eventually chaotic, in the sense of Li-Yorke and positive topological entropy.
Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascent
Inspired by this, we study standard gradient descent ascent (GDA) dynamics in a specific class of non-convex non-concave zero-sum games, that we call hidden zero-sum games.
Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games
The predominant paradigm in evolutionary game theory and more generally online learning in games is based on a clear distinction between a population of dynamic agents that interact given a fixed, static game.
Exploration-Exploitation in Multi-Agent Learning: Catastrophe Theory Meets Game Theory
Exploration-exploitation is a powerful and practical tool in multi-agent learning (MAL), however, its effects are far from understood.
Q-Learning
Computer Science and Game Theory
Multiagent Systems
Dynamical Systems
93A16, 91A26, 91A68, 58K35
G.3; J.4; F.2.2
Efficient Online Learning of Optimal Rankings: Dimensionality Reduction via Gradient Descent
We consider a natural model of online preference aggregation, where sets of preferred items $R_1, R_2, \ldots, R_t$ along with a demand for $k_t$ items in each $R_t$, appear online.
No-regret learning and mixed Nash equilibria: They do not mix
Understanding the behavior of no-regret dynamics in general $N$-player games is a fundamental question in online learning and game theory.
Chaos, Extremism and Optimism: Volume Analysis of Learning in Games
We present volume analyses of Multiplicative Weights Updates (MWU) and Optimistic Multiplicative Weights Updates (OMWU) in zero-sum as well as coordination games.
From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization
In this paper we investigate the Follow the Regularized Leader dynamics in sequential imperfect information games (IIG).
Smooth markets: A basic mechanism for organizing gradient-based learners
With the success of modern machine learning, it is becoming increasingly important to understand and control how learning algorithms interact.
Efficiently avoiding saddle points with zero order methods: No gradients required
We consider the case of derivative-free algorithms for non-convex optimization, also known as zero order algorithms, that use only function evaluations rather than gradients.
Poincaré Recurrence, Cycles and Spurious Equilibria in Gradient-Descent-Ascent for Non-Convex Non-Concave Zero-Sum Games
We study a wide class of non-convex non-concave min-max games that generalizes over standard bilinear zero-sum games.
A Peek into the Unobservable: Hidden States and Bayesian Inference for the Bitcoin and Ether Price Series
Conventional financial models fail to explain the economic and monetary properties of cryptocurrencies due to the latter's dual nature: their usage as financial assets on the one side and their tight connection to the underlying blockchain structure on the other.
Multiagent Evaluation under Incomplete Information
This paper investigates the evaluation of learned multiagent strategies in the incomplete information setting, which plays a critical role in ranking and training of agents.
Finite Regret and Cycles with Fixed Step-Size via Alternating Gradient Descent-Ascent
Gradient descent is arguably one of the most popular online optimization methods with a wide array of applications.
Computer Science and Game Theory Dynamical Systems Optimization and Control
Fast and Furious Learning in Zero-Sum Games: Vanishing Regret with Non-Vanishing Step Sizes
We show for the first time, to our knowledge, that it is possible to reconcile in online learning in zero-sum games two seemingly contradictory objectives: vanishing time-average regret and non-vanishing step sizes.
Optimistic mirror descent in saddle-point problems: Going the extra(-gradient) mile
Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond.
Multi-Agent Learning in Network Zero-Sum Games is a Hamiltonian System
Specifically, we show that no matter the size, or network structure of such closed economies, even if agents use different online learning dynamics from the standard class of Follow-the-Regularized-Leader, they yield Hamiltonian dynamics.
α-Rank: Multi-Agent Evaluation by Evolution
We introduce {\alpha}-Rank, a principled evolutionary dynamics methodology, for the evaluation and ranking of agents in large-scale multi-agent interactions, grounded in a novel dynamical game-theoretic solution concept called Markov-Conley chains (MCCs).
Short-distance commuters in the smart city
This study models and examines commuter's preferences for short-distance transportation modes, namely: walking, taking a bus or riding a metro.
Venn GAN: Discovering Commonalities and Particularities of Multiple Distributions
We propose a GAN design which models multiple distributions effectively and discovers their commonalities and particularities.
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond.
The Unusual Effectiveness of Averaging in GAN Training
We examine two different techniques for parameter averaging in GAN training.
Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaos
Interestingly, this convergence result does not carry over to the nearly homologous MWU variant where at each step the probability assigned to action $\gamma$ is multiplied by $(1 -\epsilon)^{C(\gamma)}$ even for the simplest case of two-agent, two-strategy load balancing games, where such dynamics can provably lead to limit cycles or even chaotic behavior.
First-order Methods Almost Always Avoid Saddle Points
We establish that first-order methods avoid saddle points for almost all initializations.
Cycles in adversarial regularized learning
Regularized learning is a fundamental technique in online optimization, machine learning and many other fields of computer science.
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text
Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text.
Learning Agents in Black-Scholes Financial Markets: Consensus Dynamics and Volatility Smiles
Black-Scholes (BS) is the standard mathematical model for option pricing in financial markets.
Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
Given a non-convex twice differentiable cost function f, we prove that the set of initial conditions so that gradient descent converges to saddle points where \nabla^2 f has at least one strictly negative eigenvalue has (Lebesgue) measure zero, even for cost functions f with non-isolated critical points, answering an open question in [Lee, Simchowitz, Jordan, Recht, COLT2016].
