Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions

ICLR 2019 Matthew MacKayPaul VicolJon LorraineDavid DuvenaudRoger Grosse

Hyperparameter optimization can be formulated as a bilevel optimization problem, where the optimal parameters on the training set depend on the hyperparameters. We aim to adapt regularization hyperparameters for neural networks by fitting compact approximations to the best-response function, which maps hyperparameters to optimal weights and biases... (read more)

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper