Hyperparameter Optimization

207 papers with code • 1 benchmarks • 3 datasets

Hyperparameter Optimization is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.

Source: Data-driven model for fracturing design optimization: focus on building digital database and production forecast

Libraries

Use these libraries to find Hyperparameter Optimization models and implementations
3 papers
6,646
3 papers
1,261
See all 11 libraries.

Most implemented papers

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

kubeflow/katib 21 Mar 2016

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters.

A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

fmfn/BayesianOptimization 12 Dec 2010

We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions.

Optuna: A Next-generation Hyperparameter Optimization Framework

pfnet/optuna 25 Jul 2019

We will present the design-techniques that became necessary in the development of the software that meets the above criteria, and demonstrate the power of our new design through experimental results and real world applications.

Are GANs Created Equal? A Large-Scale Study

google/compare_gan NeurIPS 2018

Generative adversarial networks (GAN) are a powerful subclass of generative models.

Optimizing Millions of Hyperparameters by Implicit Differentiation

Guang000/Awesome-Dataset-Distillation 6 Nov 2019

We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations.

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks

UKPLab/emnlp2017-bilstm-cnn-crf 21 Jul 2017

Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance.

A Tutorial on Bayesian Optimization

wujian16/Cornell-MOE 8 Jul 2018

It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample.

Single Headed Attention RNN: Stop Thinking With Your Head

Smerity/sha-rnn 26 Nov 2019

The leading approaches in language modeling are all obsessed with TV shows of my youth - namely Transformers and Sesame Street.

MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements

thomas-young-2013/lite-bo 5 Dec 2020

Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only.

Practical Bayesian Optimization of Machine Learning Algorithms

HIPS/Spearmint NeurIPS 2012

In this work, we consider the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP).