Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent
Stochastic gradient descent (SGD) with constant momentum and its variants such as Adam are the optimization algorithms of choice for training deep neural networks (DNNs). Since DNN training is incredibly computationally expensive, there is great interest in speeding up the convergence... (read more)
PDFCode
Tasks
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Nesterov Accelerated Gradient
Adam
SGD