Search Results for author:
Found 7 papers, 0 papers with code
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points
This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on the gradients.
Stochastic Gradient Methods with Preconditioned Updates
There are several algorithms for such problems, but existing methods often work poorly when the problem is badly scaled and/or ill-conditioned, and a primary goal of this work is to introduce methods that alleviate this issue.
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm
This work presents a new algorithm for empirical risk minimization.
Fast and Safe: Accelerated gradient methods with optimality certificates and underestimate sequences
In this work we introduce the concept of an Underestimate Sequence (UES), which is motivated by Nesterov's estimate sequence.
Linear Convergence of the Randomized Feasible Descent Method Under the Weak Strong Convexity Assumption
We show that the famous SDCA algorithm for optimizing the SVM dual problem, or the stochastic coordinate descent method for the LASSO problem, fits into the framework of RC-FDM.
Separable Approximations and Decomposition Methods for the Augmented Lagrangian
In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian.
Inexact Coordinate Descent: Complexity and Preconditioning
In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method.
