Search Results for author:
Found 24 papers, 7 papers with code
Analysis of Kernel Mirror Prox for Measure Optimization
By choosing a suitable function space as the dual to the non-negative measure cone, we study in a unified framework a class of functional saddle-point optimization problems, which we term the Mixed Functional Nash Equilibrium (MFNE), that underlies several existing machine learning algorithms, such as implicit generative models, distributionally robust optimization (DRO), and Wasserstein barycenters.
High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
High-probability analysis of stochastic first-order optimization methods under mild assumptions on the noise has been gaining a lot of attention in recent years.
High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance
During recent years the interest of optimization and machine learning communities in high-probability convergence of stochastic optimization methods has been growing.
A conditional gradient homotopy method with applications to Semidefinite Programming
We propose a new homotopy-based conditional gradient method for solving convex optimization problems with a large number of simple conic constraints.
Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noise
In this work, we prove the first high-probability complexity results with logarithmic dependence on the confidence level for stochastic methods for solving monotone and structured non-monotone VIPs with non-sub-Gaussian (heavy-tailed) noise and unbounded domains.
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities
We extend the stochastic extragradient method to this very general setting and theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone (when a Minty solution exists) settings.
Near-Optimal High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
In our paper, we resolve this issue and derive the first high-probability convergence results with logarithmic dependence on the confidence level for non-smooth convex stochastic optimization problems with non-sub-Gaussian (heavy-tailed) noise.
Gradient Clipping Helps in Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
In our paper, we resolve this issue and derive the first high-probability convergence results with logarithmical dependence on the confidence level for non-smooth convex stochastic optimization problems with non-sub-Gaussian (heavy-tailed) noise.
Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimization
Statistical preconditioning enables fast methods for distributed large-scale empirical risk minimization problems.
Distributed Optimization
Optimization and Control
Decentralized Distributed Optimization for Saddle Point Problems
We consider distributed convex-concave saddle point problems over arbitrary connected undirected networks and propose a decentralized distributed algorithm for their solution.
Distributed Optimization
Optimization and Control
Distributed, Parallel, and Cluster Computing
First-Order Methods for Convex Optimization
In this survey we cover a number of key developments in gradient-based optimization methods.
Tensor methods for strongly convex strongly concave saddle point problems and strongly monotone variational inequalities
The first method is based on the assumption of $p$-th order smoothness of the objective and it achieves a convergence rate of $O \left( \left( \frac{L_p R^{p - 1}}{\mu} \right)^\frac{2}{p + 1} \log \frac{\mu R^2}{\varepsilon_G} \right)$, where $R$ is an estimate of the initial distance to the solution, and $\varepsilon_G$ is the error in terms of duality gap.
Optimization and Control
Inexact Tensor Methods and Their Application to Stochastic Convex Optimization
We propose general non-accelerated and accelerated tensor methods under inexact information on the derivatives of the objective, analyze their convergence rate.
Optimization and Control
Recent Theoretical Advances in Non-Convex Optimization
For this setting, we first present known results for the convergence rates of deterministic first-order methods, which are then followed by a general theoretical analysis of optimal stochastic and randomized gradient schemes, and an overview of the stochastic first-order methods.
Zeroth-Order Algorithms for Smooth Saddle-Point Problems
In particular, our analysis shows that in the case when the feasible set is a direct product of two simplices, our convergence rate for the stochastic term is only by a $\log n$ factor worse than for the first-order methods.
Stochastic Saddle-Point Optimization for Wasserstein Barycenters
This leads to a complicated stochastic optimization problem where the objective is given as an expectation of a function given as a solution to a random optimization problem.
Accelerated meta-algorithm for convex optimization
We propose an accelerated meta-algorithm, which allows to obtain accelerated methods for convex unconstrained minimization in different settings.
Optimization and Control
Self-Concordant Analysis of Frank-Wolfe Algorithms
Projection-free optimization via different variants of the Frank-Wolfe (FW), a. k. a.
Adaptive Gradient Descent for Convex and Non-Convex Stochastic Optimization
In this paper we propose several adaptive gradient methods for stochastic optimization.
Optimization and Control
Generalized Self-concordant Hessian-barrier algorithms
Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set.
On a Combination of Alternating Minimization and Nesterov's Momentum
In this paper we combine AM and Nesterov's acceleration to propose an accelerated alternating minimization algorithm.
Distributed Computation of Wasserstein Barycenters over Networks
We propose a new \cu{class-optimal} algorithm for the distributed computation of Wasserstein Barycenters over networks.
An Accelerated Method for Derivative-Free Smooth Stochastic Convex Optimization
In the two-point feedback setting, i. e. when pairs of function values are available, we propose an accelerated derivative-free algorithm together with its complexity analysis.
Optimization and Control Computational Complexity
Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
We analyze two algorithms for approximating the general optimal transport (OT) distance between two discrete distributions of size $n$, up to accuracy $\varepsilon$.
Data Structures and Algorithms Optimization and Control
