Hyperparameter Optimization
279 papers with code • 1 benchmarks • 3 datasets
Hyperparameter Optimization is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.
Libraries
Use these libraries to find Hyperparameter Optimization models and implementations
NAS-Bench-201
NAS-Bench-101
PMLB
Latest papers
PSO-PARSIMONY: A method for finding parsimonious and accurate machine learning models with particle swarm optimization. Application for predicting force–displacement curves in T-stub steel connections
We present PSO-PARSIMONY, a new methodology to search for parsimonious and highly accurate models by means of particle swarm optimization.
HomOpt: A Homotopy-Based Hyperparameter Optimization Method
Traditional methods, like grid search and Bayesian optimization, often struggle to quickly adapt and efficiently search the loss landscape.
Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?
Hyperparameter optimization (HPO) is crucial for fine-tuning machine learning models but can be computationally expensive.
Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Search
In this work, we show that simultaneously training and mixing neural networks is a promising way to conduct Neural Architecture Search (NAS).
Multi-Objective Optimization of Performance and Interpretability of Tabular Supervised Machine Learning Models
Efficient optimization is achieved via augmentation of the search space of the learning algorithm by incorporating feature selection, interaction and monotonicity constraints into the hyperparameter search space.
PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning
Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance.
Does Long-Term Series Forecasting Need Complex Attention and Extra Long Inputs?
MABO allocates a process to each GPU via a queue mechanism, and then creates multiple trials at a time for asynchronous parallel search, which greatly reduces the search time.
Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models
aaltoml/improved-hyperparameter-learning
•
•
Approximate inference in Gaussian process (GP) models with non-conjugate likelihoods gets entangled with the learning of the model hyperparameters.
Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How
releaunifreiburg/quicktune
•
•
With the ever-increasing number of pretrained models, machine learning practitioners are continuously faced with which pretrained model to use, and how to finetune it for a new dataset.
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Recent works show that Bayesian model selection with Laplace approximations can allow to optimize such hyperparameters just like standard neural network parameters using gradients and on the training data.
