Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients

The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't... (read more)

PDF Abstract ICML 2018 PDF ICML 2018 Abstract

Tasks


Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper


METHOD TYPE
Adam
Stochastic Optimization
SGD
Stochastic Optimization