Descending through a Crowded Valley — Benchmarking Deep Learning Optimizers

1 code implementation • 1 Jan 2021 • Robin Marc Schmidt, Frank Schneider, Philipp Hennig

(iii) While we can not discern an optimization method clearly dominating across all tested tasks, we identify a significantly reduced subset of specific algorithms and parameter choices that generally lead to competitive results in our experiments.

Benchmarking