Hyperparameter Optimization

267 papers with code • 1 benchmarks • 3 datasets

Hyperparameter Optimization is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.

Source: Data-driven model for fracturing design optimization: focus on building digital database and production forecast

Libraries

Use these libraries to find Hyperparameter Optimization models and implementations
3 papers
7,343
3 papers
1,401
See all 14 libraries.

Most implemented papers

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

kubeflow/katib 21 Mar 2016

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters.

Optuna: A Next-generation Hyperparameter Optimization Framework

pfnet/optuna 25 Jul 2019

We will present the design-techniques that became necessary in the development of the software that meets the above criteria, and demonstrate the power of our new design through experimental results and real world applications.

Are GANs Created Equal? A Large-Scale Study

google/compare_gan NeurIPS 2018

Generative adversarial networks (GAN) are a powerful subclass of generative models.

Optimizing Millions of Hyperparameters by Implicit Differentiation

Guang000/Awesome-Dataset-Distillation 6 Nov 2019

We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.

A Tutorial on Bayesian Optimization

wujian16/Cornell-MOE 8 Jul 2018

It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample.

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks

UKPLab/emnlp2017-bilstm-cnn-crf 21 Jul 2017

Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance.

A System for Massively Parallel Hyperparameter Tuning

liamcli/darts_asha ICLR 2018

Modern learning models are characterized by large hyperparameter spaces and long training times.

Single Headed Attention RNN: Stop Thinking With Your Head

Smerity/sha-rnn 26 Nov 2019

The leading approaches in language modeling are all obsessed with TV shows of my youth - namely Transformers and Sesame Street.

MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements

thomas-young-2013/lite-bo 5 Dec 2020

Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only.