Search Results for author:
Found 10 papers, 5 papers with code
On the Equivalence of CoCoA+ and DisDCA
In this document, we show that the algorithm CoCoA+ (Ma et al., ICML, 2015) under the setting used in their experiments, which is also the best setting suggested by the authors that proposed this algorithm, is equivalent to the practical variant of DisDCA (Yang, NIPS, 2013).
A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization
Initial computational results on convex problems demonstrate that our method significantly improves on communication cost and running time over the current state-of-the-art methods.
A Distributed Quasi-Newton Algorithm for Primal and Dual Regularized Empirical Risk Minimization
When applied to the distributed dual ERM problem, unlike state of the art that takes only the block-diagonal part of the Hessian, our approach is able to utilize global curvature information and is thus magnitudes faster.
Accelerating Inexact Successive Quadratic Approximation for Regularized Optimization Through Manifold Identification
We show that for a wide class of degenerate solutions, ISQA+ possesses superlinear convergence not just only in iterations, but also in running time because the cost per iteration is bounded.
Optimization and Control
Momentum as Variance-Reduced Stochastic Gradient
Stochastic gradient descent with momentum (SGD+M) is widely used to empirically improve the convergence behavior and the generalization performance of plain stochastic gradient descent (SGD) in the training of deep learning models, but our theoretical understanding for SGD+M is still very limited.
Training Structured Neural Networks Through Manifold Identification and Variance Reduction
This paper proposes an algorithm (RMDA) for training neural networks (NNs) with a regularization term for promoting desired structures.
Accelerating nuclear-norm regularized low-rank matrix optimization through Burer-Monteiro decomposition
This work proposes a rapid algorithm, BM-Global, for nuclear-norm-regularized convex and low-rank matrix optimization problems.
Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network
We propose a Regularized Adaptive Momentum Dual Averaging (RAMDA) algorithm for training structured neural networks.
Manifold Identification for Ultimately Communication-Efficient Distributed Optimization
The expensive inter-machine communication is the bottleneck of distributed optimization.
