Search Results for author:
Found 42 papers, 4 papers with code
On the Optimality of Dilated Entropy and Lower Bounds for Online Learning in Extensive-Form Games
Along with a new regret lower bound for online learning in sequence-form strategy spaces, we show that this ratio is nearly optimal.
LiteEFG: An Efficient Python Library for Solving Extensive-form Games
LiteEFG is an efficient library with easy-to-use Python bindings, which can solve multiplayer extensive-form games (EFGs).
Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms
While both algorithms enjoy $O(1/T)$ ergodic convergence to Nash equilibrium in two-player zero-sum games, OMWU offers several advantages including logarithmic dependence on the size of the payoff matrix and $\widetilde{O}(1/T)$ convergence to coarse correlated equilibria even in general-sum games.
Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence Beyond the Minty Property
Policy gradient methods enjoy strong practical performance in numerous tasks in reinforcement learning.
Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning
We present a model of pragmatic language understanding, where utterances are produced and understood by searching for regularized equilibria of signaling games.
Last-Iterate Convergence Properties of Regret-Matching Algorithms in Games
Algorithms based on regret matching, specifically regret matching$^+$ (RM$^+$), and its variants are the most popular approaches for solving large-scale two-player zero-sum games in practice.
The Consensus Game: Language Model Generation via Equilibrium Search
When applied to question answering and other text generation tasks, language models (LMs) may be queried generatively (by sampling answers from their output distribution) or discriminatively (by using them to score or rank a set of candidate outputs).
The Update-Equivalence Framework for Decision-Time Planning
Using this framework, we derive a provably sound search algorithm for fully cooperative games based on mirror descent and a search algorithm for adversarial games based on magnetic mirror descent.
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Despite much progress in training AI systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge.
Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning
We then show that DiL-piKL can be extended into a self-play reinforcement learning algorithm we call RL-DiL-piKL that provides a model of human play while simultaneously training an agent that responds well to this human model.
Near-Optimal $Φ$-Regret Learning in Extensive-Form Games
In this paper, we establish efficient and uncoupled learning dynamics so that, when employed by all players in multiplayer perfect-recall imperfect-information extensive-form games, the trigger regret of each player grows as $O(\log T)$ after $T$ repetitions of play.
Near-Optimal No-Regret Learning Dynamics for General Convex Games
In this paper, we answer this in the positive by establishing the first uncoupled learning algorithm with $O(\log T)$ per-player regret in general \emph{convex games}, that is, games with concave utility functions supported on arbitrary convex and compact strategy sets.
ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret
DREAM, the only current CFR-based neural method that is model free and therefore scalable to very large games, trains a neural network on an estimated regret target that can have extremely high variance due to an importance sampling term inherited from Monte Carlo CFR (MCCFR).
Uncoupled Learning Dynamics with $O(\log T)$ Swap Regret in Multiplayer Games
In this paper we establish efficient and \emph{uncoupled} learning dynamics so that, when employed by all players in a general-sum multiplayer game, the \emph{swap regret} of each player after $T$ repetitions of the game is bounded by $O(\log T)$, improving over the prior best bounds of $O(\log^4 (T))$.
Optimal Correlated Equilibria in General-Sum Extensive-Form Games: Fixed-Parameter Algorithms, Hardness, and Two-Sided Column-Generation
We study the problem of finding optimal correlated equilibria of various sorts in extensive-form games: normal-form coarse correlated equilibrium (NFCCE), extensive-form coarse correlated equilibrium (EFCCE), and extensive-form correlated equilibrium (EFCE).
Kernelized Multiplicative Weights for 0/1-Polyhedral Games: Bridging the Gap Between Learning in Extensive-Form and Normal-Form Games
In this paper we show that the Optimistic Multiplicative Weights Update (OMWU) algorithm -- the premier learning algorithm for NFGs -- can be simulated on the normal-form equivalent of an EFG in linear time per iteration in the game tree size using a kernel trick.
Modeling Strong and Human-Like Gameplay with KL-Regularized Search
We consider the task of building strong but human-like policies in multi-agent decision-making problems, given examples of human behavior.
Equilibrium Refinement for the Age of Machines: The One-Sided Quasi-Perfect Equilibrium
In this paper, we initiate the study of equilibrium refinements for settings where one of the players is perfectly rational (the ``machine'') and the other may make mistakes.
Near-Optimal No-Regret Learning for Correlated Equilibria in Multi-Player General-Sum Games
Recently, Daskalakis, Fishelson, and Golowich (DFG) (NeurIPS`21) showed that if all agents in a multi-player general-sum normal-form game employ Optimistic Multiplicative Weights Update (OMWU), the external regret of every player is $O(\textrm{polylog}(T))$ after $T$ repetitions of the game.
Faster No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
A recent emerging trend in the literature on learning in games has been concerned with providing accelerated learning dynamics for correlated and coarse correlated equilibria in normal-form games.
Better Regularization for Sequential Decision Spaces: Fast Convergence Rates for Nash, Correlated, and Team Equilibria
The scaled extension operator is a way to recursively construct convex sets, which generalizes the decision polytope of extensive-form games, as well as the convex polytopes corresponding to correlated and team equilibria.
Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
The existence of simple uncoupled no-regret learning dynamics that converge to correlated equilibria in normal-form games is a celebrated result in the theory of multi-agent systems.
Bandit Linear Optimization for Sequential Decision Making and Extensive-Form Games
Tree-form sequential decision making (TFSDM) extends classical one-shot decision making by modeling tree-form interactions between an agent and a potentially adversarial environment.
Model-Free Online Learning in Unknown Sequential Decision Making Problems and Games
We give an efficient algorithm that achieves $O(T^{3/4})$ regret with high probability for that setting, even when the agent faces an adversarial environment.
Faster Algorithms for Optimal Ex-Ante Coordinated Collusive Strategies in Extensive-Form Zero-Sum Games
Second, we provide an algorithm that computes such an optimal distribution by only using profiles where only one of the team members gets to randomize in each profile.
Polynomial-Time Computation of Optimal Correlated Equilibria in Two-Player Extensive-Form Games with Public Chance Moves and Beyond
As of today, it is known that finding an optimal extensive-form correlated equilibrium (EFCE), extensive-form coarse correlated equilibrium (EFCCE), or normal-form coarse correlated equilibrium (NFCCE) in a two-player extensive-form game is computationally tractable when the game does not include chance moves, and intractable when the game involves chance moves.
Faster Game Solving via Predictive Blackwell Approachability: Connecting Regret Matching and Mirror Descent
In spite of this prevalence, the regret matching (RM) and regret matching+ (RM+) algorithms have been preferred in the practice of solving large-scale games (as the local regret minimizers within the counterfactual regret minimization framework).
No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
When each player has low trigger regret, the empirical frequency of play is close to an EFCE.
Stochastic Regret Minimization in Extensive-Form Games
Our framework allows us to instantiate several new stochastic methods for solving sequential games.
Efficient Regret Minimization Algorithm for Extensive-Form Correlated Equilibrium
We show that a regret minimizer can be designed for a scaled extension of any two convex sets, and that from the decomposition we then obtain a global regret minimizer.
Optimistic Regret Minimization for Extensive-Form Games via Dilated Distance-Generating Functions
Our algorithms provably converge at a rate of $T^{-1}$, which is superior to prior counterfactual regret minimization algorithms.
Coarse Correlation in Extensive-Form Games
Coarse correlation models strategic interactions of rational agents complemented by a correlation device, that is a mediator that can recommend behavior but not enforce it.
Stable-Predictive Optimistic Counterfactual Regret Minimization
The CFR framework has been a powerful tool for solving large-scale extensive-form games in practice.
Practical exact algorithm for trembling-hand equilibrium refinements in games
Nash equilibrium strategies have the known weakness that they do not prescribe rational play in situations that are reached with zero probability according to the strategies themselves, for example, if players have made mistakes.
Ex ante coordination and collusion in zero-sum multi-player extensive-form games
This paper focuses on zero-sum games where a team of players faces an opponent, as is the case, for example, in Bridge, collusion in poker, and many non-recreational applications such as war, where the colluders do not have time or means of communicating during battle, collusion in bidding, where communication during the auction is illegal, and coordinated swindling in public.
Regret Circuits: Composability of Regret Minimizers
We show that local regret minimizers for the simpler sets can be combined with additional regret minimizers into an aggregate regret minimizer for the composite set.
Solving Large Sequential Games with the Excessive Gap Technique
We present, to our knowledge, the first GPU implementation of a first-order method for extensive-form games.
Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games
Experiments show that our framework leads to algorithms that scale at a rate comparable to the fastest variants of counterfactual regret minimization for computing Nash equilibrium, and therefore our approach leads to the first algorithm for computing quantal response equilibria in extremely large games.
Robust Stackelberg Equilibria in Extensive-Form Games and Extension to Limited Lookahead
We then extend the program to the robust setting for Stackelberg equilibrium under unlimited and under limited lookahead by the opponent.
Regret Minimization in Behaviorally-Constrained Zero-Sum Games
We use an instantiation of the CFR framework to develop algorithms for solving behaviorally-constrained (and, as a special case, perturbed in the Selten sense) extensive-form games, which allows us to compute approximate Nash equilibrium refinements.
Operation Frames and Clubs in Kidney Exchange
A kidney exchange is a centrally-administered barter market where patients swap their willing yet incompatible donors.
Decoding Hidden Markov Models Faster Than Viterbi Via Online Matrix-Vector (max, +)-Multiplication
In this paper, we present a novel algorithm for the maximum a posteriori decoding (MAPD) of time-homogeneous Hidden Markov Models (HMM), improving the worst-case running time of the classical Viterbi algorithm by a logarithmic factor.
