Search Results for author:
Found 68 papers, 1 papers with code
Gradient-free Online Learning in Continuous Games with Delayed Rewards
Motivated by applications to online advertising and recommender systems, we consider a game-theoretic model with delayed rewards and asynchronous, payoff-based feedback.
Accelerated regularized learning in finite N-person games
Motivated by the success of Nesterov's accelerated gradient algorithm for convex minimization problems, we examine whether it is possible to achieve similar performance gains in the context of online learning in games.
No-regret learning in harmonic games: Extrapolation in the face of conflicting interests
By contrast, in harmonic games - the strategic counterpart of potential games, where players have conflicting interests - very little is known outside the narrow subclass of 2-player zero-sum games with a fully-mixed equilibrium.
Nested replicator dynamics, nested logit choice, and similarity-based learning
We consider a model of learning and evolution in games whose action sets are endowed with a partition-based similarity structure intended to capture exogenous similarities between strategies.
What is the long-run distribution of stochastic gradient descent? A large deviations analysis
Using an approach based on the theory of large deviations and randomly perturbed dynamical systems, we show that the long-run distribution of SGD resembles the Boltzmann-Gibbs distribution of equilibrium thermodynamics with temperature equal to the method's step-size and energy levels determined by the problem's objective and the statistics of the noise.
Tamed Langevin sampling under weaker conditions
Motivated by applications to deep learning which often fail standard Lipschitz smoothness requirements, we examine the problem of sampling from distributions that are not log-concave and are only weakly dissipative, with log-gradients allowed to grow superlinearly at infinity.
A geometric decomposition of finite games: Convergence vs. recurrence under exponential weights
In view of the complexity of the dynamics of learning in games, we seek to decompose a game into simpler components where the dynamics' long-run behavior is well understood.
Exploiting hidden structures in non-convex games for convergence to Nash equilibrium
A wide array of modern machine learning applications - from adversarial models to multi-agent reinforcement learning - can be formulated as non-cooperative games whose Nash equilibria represent the system's desired operational states.
A Quadratic Speedup in Finding Nash Equilibria of Quantum Zero-Sum Games
In 2008, Jain and Watrous proposed the first classical algorithm for computing equilibria in quantum zero-sum games using the Matrix Multiplicative Weight Updates (MMWU) method to achieve a convergence rate of $\mathcal{O}(d/\epsilon^2)$ iterations to $\epsilon$-Nash equilibria in the $4^d$-dimensional spectraplex.
Learning in quantum games
In this paper, we introduce a class of learning dynamics for general quantum games, that we call "follow the quantum regularized leader" (FTQL), in reference to the classical "follow the regularized leader" (FTRL) template for learning in finite games.
The rate of convergence of Bregman proximal methods: Local geometry vs. regularity vs. sharpness
For generality, we focus on local solutions of constrained, non-monotone variational inequalities, and we show that the convergence rate of a given method depends sharply on its associated Legendre exponent, a notion that measures the growth rate of the underlying Bregman function (Euclidean, entropic, or other) near a solution.
Explicit Second-Order Min-Max Optimization Methods with Optimal Convergence Guarantee
Specifically, we show that the proposed methods generate iterates that remain within a bounded set and that the averaged iterates converge to an $\epsilon$-saddle point within $O(\epsilon^{-2/3})$ iterations in terms of a restricted gap function.
On the convergence of policy gradient methods to Nash equilibria in general stochastic games
Learning in stochastic games is a notoriously difficult problem because, in addition to each other's strategic decisions, the players must also contend with the fact that the game itself evolves over time, possibly in a very complicated manner.
Pick your Neighbor: Local Gauss-Southwell Rule for Fast Asynchronous Decentralized Optimization
In decentralized optimization environments, each agent $i$ in a network of $n$ nodes has its own private function $f_i$, and nodes communicate with their neighbors to cooperatively minimize the aggregate objective $\sum_{i=1}^n f_i$.
Nested bandits
In many online decision processes, the optimizing agent is called to choose between large numbers of alternatives with many inherent similarities; in turn, these similarities imply closely correlated losses that may confound standard discrete choice models and bandit algorithms.
Riemannian stochastic approximation algorithms
We examine a wide class of stochastic approximation algorithms for solving (stochastic) nonlinear problems on Riemannian manifolds.
No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation
We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully adversarial environments.
A unified stochastic approximation framework for learning in games
We develop a flexible stochastic approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite).
On the Rate of Convergence of Regularized Learning in Games: From Bandits and Uncertainty to Optimism and Beyond
In this paper, we examine the convergence rate of a wide range of regularized methods for learning in games.
Fast Routing under Uncertainty: Adaptive Learning in Congestion Games via Exponential Weights
We examine an adaptive learning framework for nonatomic congestion games where the players' cost functions may be subject to exogenous fluctuations (e. g., due to disturbances in the network, variations in the traffic going through a link).
Sifting through the noise: Universal first-order methods for stochastic variational inequalities
Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.
Zeroth-order non-convex learning via hierarchical dual averaging
We propose a hierarchical version of dual averaging for zeroth-order online non-convex optimization - i. e., learning processes where, at each stage, the optimizer is facing an unknown non-convex loss function and only receives the incurred loss as feedback.
Adaptive first-order methods revisited: Convex optimization without Lipschitz requirements
We propose a new family of adaptive first-order methods for a class of convex minimization problems that may fail to be Lipschitz continuous or smooth in the standard sense.
Distributed stochastic optimization with large delays
One of the most widely used methods for solving large-scale stochastic optimization problems is distributed asynchronous stochastic gradient descent (DASGD), a family of algorithms that result from parallelizing stochastic gradient descent on distributed computing architectures (possibly) asychronously.
The Last-Iterate Convergence Rate of Optimistic Mirror Descent in Stochastic Variational Inequalities
In this paper, we analyze the local convergence rate of optimistic mirror descent methods in stochastic variational inequalities, a class of optimization problems with important applications to learning theory and machine learning.
Learning in nonatomic games, Part I: Finite action spaces and population games
We examine the long-run behavior of a wide range of dynamics for learning in nonatomic games, in both discrete and continuous time.
Optimization in Open Networks via Dual Averaging
In networks of autonomous agents (e. g., fleets of vehicles, scattered sensors), the problem of minimizing the sum of the agents' local functions has received a lot of interest.
Adaptive First-Order Methods Revisited: Convex Minimization without Lipschitz Requirements
We propose a new family of adaptive first-order methods for a class of convex minimization problems that may fail to be Lipschitz continuous or smooth in the standard sense.
Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibrium
In game-theoretic learning, several agents are simultaneously following their individual interests, so the environment is non-stationary from each player's perspective.
Survival of the strictest: Stable and unstable equilibria under regularized learning with partial information
This equivalence extends existing continuous-time versions of the folk theorem of evolutionary game theory to a bona fide algorithmic learning setting, and it provides a clear refinement criterion for the prediction of the day-to-day behavior of no-regret learning in games
Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism
In this paper, we provide a general framework for studying multi-agent online learning problems in the presence of delays and asynchronicities.
Adaptive extra-gradient methods for min-max optimization and games
We present a new family of min-max optimization algorithms that automatically exploit the geometry of the gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones.
No-regret learning and mixed Nash equilibria: They do not mix
Understanding the behavior of no-regret dynamics in general $N$-player games is a fundamental question in online learning and game theory.
Online non-convex optimization with imperfect feedback
We consider the problem of online learning with non-convex losses.
Regret minimization in stochastic non-convex learning via a proximal-gradient approach
In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping.
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.
The limits of min-max optimization algorithms: convergence to spurious non-critical sets
Compared to ordinary function minimization problems, min-max optimization algorithms encounter far greater challenges because of the existence of periodic cycles and similar phenomena.
Online and stochastic optimization beyond Lipschitz continuity: A Riemannian approach
Motivated by applications to machine learning and imaging science, we study a class of online and stochastic optimization problems with loss functions that are not Lipschitz continuous; in particular, the loss functions encountered by the optimizer could exhibit gradient singularities or be singular themselves.
Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
Owing to their stability and convergence speed, extragradient methods have become a staple for solving large-scale saddle-point problems in machine learning.
A new regret analysis for Adam-type algorithms
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).
Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games
In this paper, we consider multi-agent learning via online gradient descent in a class of games called $\lambda$-cocoercive games, a fairly broad class of games that admits many Nash equilibria and that properly includes unconstrained strongly monotone games.
An adaptive Mirror-Prox method for variational inequalities with singular operators
Lipschitz continuity is a central requirement for achieving the optimal O(1/T) rate of convergence in monotone, deterministic variational inequalities (a setting that includes convex minimization, convex-concave optimization, nonatomic games, and many other problems).
On the convergence of single-call stochastic extra-gradient methods
Variational inequalities have recently attracted considerable interest in machine learning as a flexible paradigm for models that go beyond ordinary loss function minimization (such as generative adversarial networks and related deep learning systems).
Optimistic mirror descent in saddle-point problems: Going the extra(-gradient) mile
Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond.
Forward-backward-forward methods with variance reduction for stochastic variational inequalities
We develop a new stochastic algorithm with variance reduction for solving pseudo-monotone stochastic variational inequalities.
Learning in Games with Lossy Feedback
We consider a game-theoretical multi-agent learning problem where the feedback information can be lost during the learning process and rewards are given by a broad class of games known as variationally stable games.
Bandit Learning in Concave N-Person Games
This paper examines the long-run behavior of learning with bandit feedback in non-cooperative concave games.
Bandit learning in concave $N$-person games
This paper examines the long-run behavior of learning with bandit feedback in non-cooperative concave games.
Hessian barrier algorithms for linearly constrained optimization problems
In the case of linearly constrained quadratic programs (not necessarily convex), we also show that the method's convergence rate is $\mathcal{O}(1/k^\rho)$ for some $\rho\in(0, 1]$ that depends only on the choice of kernel function (i. e., not on the problem's primitives).
Multi-agent online learning in time-varying games
We examine the long-run behavior of multi-agent online learning in games that evolve over time.
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile
Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond.
Distributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go?
One of the most widely used optimization methods for large-scale machine learning problems is distributed asynchronous stochastic gradient descent (DASGD).
Online convex optimization and no-regret learning: Algorithms, guarantees and applications
Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role.
Stochastic Mirror Descent in Variationally Coherent Optimization Problems
In this paper, we examine a class of non-convex stochastic optimization problems which we call variationally coherent, and which properly includes pseudo-/quasiconvex and star-convex optimization problems.
Learning with Bandit Feedback in Potential Games
This paper examines the equilibrium convergence properties of no-regret learning with exponential weights in potential games.
Countering Feedback Delays in Multi-Agent Learning
We consider a model of game-theoretic learning based on online mirror descent (OMD) with asynchronous and delayed feedback information.
Cycles in adversarial regularized learning
Regularized learning is a fundamental technique in online optimization, machine learning and many other fields of computer science.
On the convergence of mirror descent beyond stochastic convex programming
In this paper, we examine the convergence of mirror descent in a class of stochastic optimization problems that are not necessarily convex (or even quasi-convex), and which we call variationally coherent.
On the convergence of gradient-like flows with noisy gradient input
In the vanishing noise limit, we show that the dynamics converge to the solution set of the underlying problem (a. s.).
Learning in games with continuous action sets and unknown payoff functions
This paper examines the convergence of no-regret learning in games with continuous action sets.
Exponentially fast convergence to (strict) equilibrium via hedging
Motivated by applications to data networks where fast convergence is essential, we analyze the problem of learning in generic N-person games that admit a Nash equilibrium in pure strategies.
Distributed stochastic optimization via matrix exponential learning
In this paper, we investigate a distributed learning scheme for a broad class of stochastic optimization problems and games that arise in signal processing and wireless communications.
Boltzmann meets Nash: Energy-efficient routing in optical networks under uncertainty
Motivated by the massive deployment of power-hungry data centers for service provisioning, we examine the problem of routing in optical networks with the aim of minimizing traffic-driven power consumption.
On the robustness of learning in games with stochastically perturbed payoff observations
Motivated by the scarcity of accurate payoff feedback in practical applications of game theory, we examine a class of learning dynamics where players adjust their choices based on past payoff observations that are subject to noise and random disturbances.
Game-theoretical control with continuous action sets
To do so, we extend the theory of finite-dimensional two-timescale stochastic approximation to an infinite-dimensional, Banach space setting, and we prove that the continuous dynamics of the process converge to equilibrium in the case of potential games.
Learning in games via reinforcement and regularization
We investigate a class of reinforcement learning dynamics where players adjust their strategies based on their actions' cumulative payoffs over time - specifically, by playing mixed strategies that maximize their expected cumulative payoff minus a regularization term.
A continuous-time approach to online optimization
We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret.
Penalty-regulated dynamics and robust learning procedures in games
Starting from a heuristic learning scheme for N-person games, we derive a new class of continuous-time learning dynamics consisting of a replicator-like drift adjusted by a penalty term that renders the boundary of the game's strategy space repelling.
