Search Results for author:
Found 5 papers, 0 papers with code
SignSVRG: fixing SignSGD via variance reduction
The core idea is first instantiated on the problem of minimizing sums of convex and Lipschitz functions and is then extended to the smooth case via variance reduction.
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold
We consider the problem of minimizing a non-convex function over a smooth manifold $\mathcal{M}$.
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
First, we formulate the training of quantized neural networks (QNNs) as a smoothed sequence of interval-constrained optimization problems.
Stochastic Subgradient Descent on a Generic Definable Function Converges to a Minimizer
It was previously shown by Davis and Drusvyatskiy that every Clarke critical point of a generic, semialgebraic (and more generally definable in an o-minimal structure), weakly convex function is lying on an active manifold and is either a local minimum or an active strict saddle.
Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions
Consequently, generically in the class of definable weakly convex functions, the SGD converges to a local minimizer.
