Search Results for author:
Found 8 papers, 2 papers with code
LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace
As a result, the constructed subspace is able to dynamically and incrementally approximate the Hessian inverse vector product with less effort and thus leads to a favorable estimate of the hyper-gradient.
Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity
A significant milestone in modern gradient-based optimization was achieved with the development of Nesterov's accelerated gradient descent (NAG) method.
On Underdamped Nesterov's Acceleration
In this paper, based on the high-resolution differential equation framework, we construct the new Lyapunov functions for the underdamped case, which is motivated by the power of the time $t^{\gamma}$ or the iteration $k^{\gamma}$ in the mixed term.
Linear Convergence of ISTA and FISTA
Specifically, assuming the smooth part to be strongly convex is more reasonable for the least-square model, even though the image matrix is probably ill-conditioned.
Revisiting the acceleration phenomenon via high-resolution differential equations
Furthermore, we also investigate NAG from the implicit-velocity scheme.
Proximal Subgradient Norm Minimization of ISTA and FISTA
We apply the tighter inequality discovered in the well-constructed Lyapunov function and then obtain the proximal subgradient norm minimization by the phase-space representation, regardless of gradient-correction or implicit-velocity.
Gradient Norm Minimization of Nesterov Acceleration: $o(1/k^3)$
In the history of first-order algorithms, Nesterov's accelerated gradient descent (NAG) is one of the milestones.
Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints
Numerical experiments in serial illustrate that the novel updating rule for the Lagrangian multipliers significantly accelerates the convergence of PLAM and makes it comparable with the existent feasible solvers for optimization problems with orthogonality constraints, and the performance of PCAL does not highly rely on the choice of the penalty parameter.
Optimization and Control 15A18, 65F15, 65K05, 90C06
