Search Results for author:
Found 10 papers, 3 papers with code
Adaptive Proximal Gradient Method for Convex Optimization
In this paper, we explore two fundamental first-order algorithms in convex optimization, namely, gradient descent (GD) and proximal gradient method (ProxGD).
Beyond the Golden Ratio for Variational Inequality Algorithms
We improve the understanding of the $\textit{golden ratio algorithm}$, which solves monotone variational inequalities (VI) and convex-concave min-max problems via the distinctive feature of adapting the step sizes to the local Lipschitz constants.
Convergence of adaptive algorithms for constrained weakly convex optimization
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
A first-order primal-dual method with adaptivity to local smoothness
We consider the problem of finding a saddle point for the convex-concave objective $\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)$, where $f$ is a convex function with locally Lipschitz gradient and $g$ is convex and possibly non-smooth.
Stochastic Variance Reduction for Variational Inequality Methods
We propose stochastic variance reduced algorithms for solving convex-concave saddle point problems, monotone variational inequalities, and monotone inclusions.
Convergence of adaptive algorithms for weakly convex constrained optimization
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
A new regret analysis for Adam-type algorithms
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).
Adaptive Gradient Descent without Descent
We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don't increase the stepsize too fast and 2) don't overstep the local curvature.
Revisiting Stochastic Extragradient
We fix a fundamental issue in the stochastic extragradient method by providing a new sampling strategy that is motivated by approximating implicit updates.
Model Function Based Conditional Gradient Method with Armijo-like Line Search
The Conditional Gradient Method is generalized to a class of non-smooth non-convex optimization problems with many applications in machine learning.
