Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?

27 Feb 2020

It is well-known that given a bounded, smooth nonconvex function, standard gradient-based methods can find $\epsilon$-stationary points (where the gradient norm is less than $\epsilon$) in $\mathcal{O}(1/\epsilon^2)$ iterations. However, many important nonconvex optimization problems, such as those associated with training modern neural networks, are inherently not smooth, making these results inapplicable... (read more)

PDF Abstract

Code Add Remove Mark official

No code implementations yet. Submit your code now