Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions

3 code implementations • ICLR 2019 • Matthew MacKay, Paul Vicol, Jon Lorraine, David Duvenaud, Roger Grosse

Empirically, our approach outperforms competing hyperparameter optimization methods on large-scale deep learning problems.

Bilevel Optimization Data Augmentation +1

Reversible Recurrent Neural Networks

1 code implementation • NeurIPS 2018 • Matthew MacKay, Paul Vicol, Jimmy Ba, Roger Grosse

Reversible RNNs---RNNs for which the hidden-to-hidden transition can be reversed---offer a path to reduce the memory requirements of training, as hidden states need not be stored and instead can be recomputed during backpropagation.