Hyperparameter Optimization
278 papers with code • 1 benchmarks • 3 datasets
Hyperparameter Optimization is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.
Libraries
Use these libraries to find Hyperparameter Optimization models and implementations
NAS-Bench-201
NAS-Bench-101
PMLB
Most implemented papers
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
kubeflow/katib
•
•
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters.
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions.
Optuna: A Next-generation Hyperparameter Optimization Framework
pfnet/optuna
•
•
We will present the design-techniques that became necessary in the development of the software that meets the above criteria, and demonstrate the power of our new design through experimental results and real world applications.
Are GANs Created Equal? A Large-Scale Study
google/compare_gan
•
•
NeurIPS 2018
Generative adversarial networks (GAN) are a powerful subclass of generative models.
Optimizing Millions of Hyperparameters by Implicit Differentiation
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.
A Tutorial on Bayesian Optimization
It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample.
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
UKPLab/emnlp2017-bilstm-cnn-crf
•
•
Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance.
A System for Massively Parallel Hyperparameter Tuning
liamcli/darts_asha
•
•
ICLR 2018
Modern learning models are characterized by large hyperparameter spaces and long training times.
Single Headed Attention RNN: Stop Thinking With Your Head
Smerity/sha-rnn
•
•
The leading approaches in language modeling are all obsessed with TV shows of my youth - namely Transformers and Sesame Street.
MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements
Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only.
