LRTuner: A Learning Rate Tuner for Deep Neural Networks

1 code implementation • ICML Workshop AutoML 2021 • Nikhil Iyer, V Thejas, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu

For example on ImageNet with Resnet-50, LRTuner shows up to 0. 2% absolute gains in test accuracy compared to the hand-tuned baseline schedule.

Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule

2 code implementations • 9 Mar 2020 • Nikhil Iyer, V Thejas, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu

Several papers argue that wide minima generalize better than narrow minima.

de-en Machine Translation

AutoLR: A Method for Automatic Tuning of Learning Rate

no code implementations • 25 Sep 2019 • Nipun Kwatra, V Thejas, Nikhil Iyer, Ramachandran Ramjee, Muthian Sivathanu

We compare favorably against state of the art learning rate schedules for the given dataset and models, including for ImageNet on Resnet-50, Cifar-10 on Resnet-18, and SQuAD fine-tuning on BERT.