Search Results for author:
Found 2 papers, 0 papers with code
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
The Adaptive Momentum Estimation (Adam) algorithm is highly effective in training various deep learning tasks.
High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise
To overcome these limitations, we provide a deep analysis and show that Adam could converge to the stationary point in high probability with a rate of $\mathcal{O}\left({\rm poly}(\log T)/\sqrt{T}\right)$ under coordinate-wise "affine" variance noise, not requiring any bounded gradient assumption and any problem-dependent knowledge in prior to tune hyper-parameters.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
