Search Results for author:
Found 34 papers, 4 papers with code
Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback
Online gradient descent (OGD) is well known to be doubly optimal under strong convexity or monotonicity assumptions: (1) in the single-agent setting, it achieves an optimal regret of $\Theta(\log T)$ for strongly convex cost functions; and (2) in the multi-agent setting of strongly monotone games, with each agent employing OGD, we obtain last-iterate convergence of the joint action to a unique Nash equilibrium at an optimal rate of $\Theta(\frac{1}{T})$.
A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport
Kernel-based optimal transport (OT) estimators offer an alternative, functional estimation procedure to address OT problems from samples.
Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds
Numerous applications in machine learning and data analytics can be formulated as equilibrium computation over Riemannian manifolds.
Deterministic Nonsmooth Nonconvex Optimization
In particular, we prove a lower bound of $\Omega(d)$ for any deterministic algorithm.
Explicit Second-Order Min-Max Optimization Methods with Optimal Convergence Guarantee
We propose and analyze exact and inexact regularized Newton-type methods for finding a global saddle point of \emph{convex-concave} unconstrained min-max optimization problems.
Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization
Nonsmooth nonconvex optimization problems broadly emerge in machine learning and business decision making, whereas two core challenges impede the development of efficient solution methods with finite-time convergence guarantee: the lack of computationally tractable optimality criterion and the lack of computationally powerful oracles.
Understanding Performance of Long-Document Ranking Models through Comprehensive Evaluation and Leaderboarding
Most other models had poor zero-shot performance (sometimes at a random baseline level) but outstripped MaxP by as much 13-28\% after finetuning.
First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces
From optimal transport to robust dimensionality reduction, a plethora of machine learning applications can be cast into the min-max optimization problems over Riemannian manifolds.
Online Nonsubmodular Minimization with Delayed Costs: From Full Information to Bandit Feedback
Motivated by applications to online learning in sparse estimation and Bayesian optimization, we consider the problem of online unconstrained nonsubmodular minimization with delayed costs in both full information and bandit feedback settings.
Perseus: A Simple and Optimal High-Order Method for Variational Inequalities
Our method with restarting attains a linear rate for smooth and uniformly monotone VIs and a local superlinear rate for smooth and strongly monotone VIs.
First-Order Algorithms for Nonlinear Generalized Nash Equilibrium Problems
We consider the problem of computing an equilibrium in a class of \textit{nonlinear generalized Nash equilibrium problems (NGNEPs)} in which the strategy sets for each player are defined by equality and inequality constraints that may depend on the choices of rival players.
Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedback
Leveraging self-concordant barrier functions, we first construct a new bandit learning algorithm and show that it achieves the single-agent optimal regret of $\tilde{\Theta}(n\sqrt{T})$ under smooth and strongly concave reward functions ($n \geq 1$ is the problem dimension).
Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization
Distributionally robust supervised learning (DRSL) is emerging as a key paradigm for building reliable machine learning systems for real-world applications -- reflecting the need for classifiers and predictive models that are robust to the distribution shifts that arise from phenomena such as selection bias or nonstationarity.
A Variational Inequality Approach to Bayesian Regression Games
Bayesian regression games are a special class of two-player general-sum Bayesian games in which the learner is partially informed about the adversary's objective through a Bayesian prior.
On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification
Yet, the behavior of minimum Wasserstein estimators is poorly understood, notably in high-dimensional regimes or under model misspecification.
Projection Robust Wasserstein Distance and Riemannian Optimization
Projection robust Wasserstein (PRW) distance, or Wasserstein projection pursuit (WPP), is a robust variant of the Wasserstein distance.
Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games
In this paper, we consider multi-agent learning via online gradient descent in a class of games called $\lambda$-cocoercive games, a fairly broad class of games that admits many Nash equilibria and that properly includes unconstrained strongly monotone games.
Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm
We study the fixed-support Wasserstein barycenter problem (FS-WBP), which consists in computing the Wasserstein barycenter of $m$ discrete probability measures supported on a finite metric space of size $n$.
Near-Optimal Algorithms for Minimax Optimization
This paper presents the first algorithm with $\tilde{O}(\sqrt{\kappa_{\mathbf x}\kappa_{\mathbf y}})$ gradient complexity, matching the lower bound up to logarithmic factors.
On the Complexity of Approximating Multimarginal Optimal Transport
This provides a first \textit{near-linear time} complexity bound guarantee for approximating the MOT problem and matches the best known complexity bound for the Sinkhorn algorithm in the classical OT setting when $m = 2$.
On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
We consider nonconvex-concave minimax problems, $\min_{\mathbf{x}} \max_{\mathbf{y} \in \mathcal{Y}} f(\mathbf{x}, \mathbf{y})$, where $f$ is nonconvex in $\mathbf{x}$ but concave in $\mathbf{y}$ and $\mathcal{Y}$ is a convex and bounded set.
On the Efficiency of Entropic Regularized Algorithms for Optimal Transport
We prove that APDAMD achieves the complexity bound of $\widetilde{O}(n^2\sqrt{\delta}\varepsilon^{-1})$ in which $\delta>0$ stands for the regularity of $\phi$.
On Structured Filtering-Clustering: Global Error Bound and Optimal First-Order Algorithms
We also conduct experiments on real datasets and the numerical results demonstrate the effectiveness of our algorithms.
On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms
We show that a greedy variant of the classical Sinkhorn algorithm, known as the \emph{Greenkhorn algorithm}, can be improved to $\widetilde{\mathcal{O}}(n^2\varepsilon^{-2})$, improving on the best known complexity bound of $\widetilde{\mathcal{O}}(n^2\varepsilon^{-3})$.
Data Structures and Algorithms
Sparsemax and Relaxed Wasserstein for Topic Sparsity
As topic sparsity of individual documents in online social media increases, so does the difficulty of analyzing the online text sources using traditional methods.
Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradient
Convex composition optimization is an emerging topic that covers a wide range of applications arising from stochastic optimal control, reinforcement learning and multi-stage stochastic programming.
An ADMM-Based Interior-Point Method for Large-Scale Linear Programming
Due its connection to Newton's method, IPM is often classified as second-order method -- a genre that is attached with stability and accuracy at the expense of scalability.
Optimization and Control
Improved Oracle Complexity of Variance Reduced Methods for Nonsmooth Convex Stochastic Composition Optimization
We consider the nonsmooth convex composition optimization problem where the objective is a composition of two finite-sum functions and analyze stochastic compositional variance reduced gradient (SCVRG) methods for them.
On the Iteration Complexity Analysis of Stochastic Primal-Dual Hybrid Gradient Approach with High Probability
In this paper, we propose a stochastic Primal-Dual Hybrid Gradient (PDHG) approach for solving a wide spectrum of regularized stochastic minimization problems, where the regularization term is composite with a linear function.
Stochastic Primal-Dual Proximal ExtraGradient Descent for Compositely Regularized Optimization
This optimization model abstracts a number of important applications in artificial intelligence and machine learning, such as fused Lasso, fused logistic regression, and a class of graph-guided regularized minimization.
Relaxed Wasserstein with Applications to GANs
Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications.
Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis
In particular, we consider in this paper some constrained nonconvex optimization models in block decision variables, with or without coupled affine constraints.
Global Convergence of Unmodified 3-Block ADMM for a Class of Convex Minimization Problems
The alternating direction method of multipliers (ADMM) has been successfully applied to solve structured convex optimization problems due to its superior practical performance.
An Extragradient-Based Alternating Direction Method for Convex Minimization
The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings.
