Hyperparameter Optimization
278 papers with code • 1 benchmarks • 3 datasets
Hyperparameter Optimization is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.
Libraries
Use these libraries to find Hyperparameter Optimization models and implementations
NAS-Bench-201
NAS-Bench-101
PMLB
Latest papers
Intelligent Learning Rate Distribution to reduce Catastrophic Forgetting in Transformers
themody/nas-catastrophicforgetting
•
•
We combine the learning rate distributions thus found and show that they generalize to better performance with respect to the problem of catastrophic forgetting.
Breast Cancer Classification Using Gradient Boosting Algorithms Focusing on Reducing the False Negative and SHAP for Explainability
The main objective of this study is to use state-of-the-art boosting algorithms such as AdaBoost, XGBoost, CatBoost and LightGBM to predict and diagnose breast cancer and to find the most effective metric regarding recall, ROC-AUC, and confusion matrix.
FeatAug: Automatic Feature Augmentation From One-to-Many Relationship Tables
sfu-db/feataug
•
•
To overcome this limitation, we propose FEATAUG, a new feature augmentation framework that automatically extracts predicate-aware SQL queries from one-to-many relationship tables.
Better Understandings and Configurations in MaxSAT Local Search Solvers via Anytime Performance Analysis
Though numerous solvers have been proposed for the MaxSAT problem, and the benchmark environment such as MaxSAT Evaluations provides a platform for the comparison of the state-of-the-art solvers, existing assessments were usually evaluated based on the quality, e. g., fitness, of the best-found solutions obtained within a given running time budget.
Rethinking of Encoder-based Warm-start Methods in Hyperparameter Optimization
azoz01/liltab
•
•
In this work, we evaluate Dataset2Vec and liltab on two common meta-tasks - representing entire datasets and hyperparameter optimization warm-start.
Hyperparameter Tuning MLPs for Probabilistic Time Series Forecasting
Time series forecasting attempts to predict future events by analyzing past trends and patterns.
Parallel Hyperparameter Optimization Of Spiking Neural Network
By defining an early stopping criterion detecting silent networks and by designing specific constraints, we were able to instantiate larger and more flexible search spaces.
Explainable Bayesian Optimization
In industry, Bayesian optimization (BO) is widely applied in the human-AI collaborative parameter tuning of cyber-physical systems.
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
mingruiliu-ml-lab/bilevel-optimization-under-unbounded-smoothness
•
•
When the upper-level problem is nonconvex and unbounded smooth, and the lower-level problem is strongly convex, we prove that our algorithm requires $\widetilde{\mathcal{O}}(1/\epsilon^4)$ iterations to find an $\epsilon$-stationary point in the stochastic setting, where each iteration involves calling a stochastic gradient or Hessian-vector product oracle.
Teaching Specific Scientific Knowledge into Large Language Models through Additional Training
Through additional training, we explore embedding specialized scientific knowledge into the Llama 2 Large Language Model (LLM).
