Search Results for author:
Found 14 papers, 1 papers with code
Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
Empirical results on deep learning verify that when adopting the same large batch size, SNGM can achieve better test accuracy than MSGD and other state-of-the-art large-batch training methods.
Stagewise Enlargement of Batch Size for SGD-based Learning
We theoretically prove that, compared to classical stagewise SGD which decreases learning rate by stage, \mbox{SEBS} can reduce the number of parameter updates without increasing generalization error.
ADASS: Adaptive Sample Selection for Training Acceleration
However, in all existing SGD and its variants, the sample size in each iteration~(epoch) of training is the same as the size of the full training set.
Clustered Reinforcement Learning
Exploration strategy design is one of the challenging problems in reinforcement learning~(RL), especially when the environment contains a large state space or sparse rewards.
Global Momentum Compression for Sparse Communication in Distributed Learning
With the rapid growth of data, distributed momentum stochastic gradient descent~(DMSGD) has been widely used in distributed learning, especially for training large-scale deep models.
On the Convergence of Memory-Based Distributed SGD
Using the transformation equation, we propose the convergence rate of stagewise M-DSGD which bridges the gap between theory and practice.
Quantized Epoch-SGD for Communication-Efficient Distributed Learning
Due to its efficiency and ease to implement, stochastic gradient descent (SGD) has been widely used in machine learning.
Proximal SCOPE for Distributed Sparse Learning
Based on the defined metric, we theoretically prove that pSCOPE is convergent with a linear convergence rate if the data partition is good enough.
Proximal SCOPE for Distributed Sparse Learning: Better Data Partition Implies Faster Convergence Rate
Based on the defined metric, we theoretically prove that pSCOPE is convergent with a linear convergence rate if the data partition is good enough.
Feature-Distributed SVRG for High-Dimensional Linear Classification
Linear classification has been widely used in many high-dimensional applications like text classification.
SCOPE: Scalable Composite Optimization for Learning on Spark
Recently, many distributed stochastic optimization~(DSO) methods have been proposed to solve the large-scale composite optimization problems, which have shown better performance than traditional batch methods.
Fast Asynchronous Parallel Stochastic Gradient Decent
Stochastic gradient descent~(SGD) and its variants have become more and more popular in machine learning due to their efficiency and effectiveness.
Scalable Stochastic Alternating Direction Method of Multipliers
There exists only one stochastic method, called SA-ADMM, which can achieve convergence rate $O(1/T)$ on general convex problems.
