Search Results for author:
Found 26 papers, 8 papers with code
Structural Pruning of Pre-trained Language Models via Neural Architecture Search
Pre-trained language models (PLM), for example BERT or RoBERTa, mark the state-of-the-art for natural language understanding task when fine-tuned on labeled data.
A Negative Result on Gradient Matching for Selective Backprop
With increasing scale in model and dataset size, the training of deep neural networks becomes a massive computational burden.
Optimizing Hyperparameters with Conformal Quantile Regression
Many state-of-the-art hyperparameter optimization (HPO) algorithms rely on model-based optimizers that learn surrogate models of the target function to guide the search.
Renate: A Library for Real-World Continual Learning
We discuss requirements for the use of continual learning algorithms in practice, from which we derive design principles for Renate.
Fortuna: A Library for Uncertainty Quantification in Deep Learning
We present Fortuna, an open-source library for uncertainty quantification in deep learning.
Private Synthetic Data for Multitask Learning and Marginal Queries
A key innovation in our algorithm is the ability to directly handle numerical features, in contrast to a number of related prior approaches which require numerical features to be first converted into {high cardinality} categorical features via {a binning strategy}.
Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priors
As we move away from the data, the predictive uncertainty should increase, since a great variety of explanations are consistent with the little available information.
Continual Learning with Transformers for Image Classification
This phenomenon is known as catastrophic forgetting and it is often difficult to prevent due to practical constraints, such as the amount of data that can be stored or the limited computation sources that can be used.
Diverse Counterfactual Explanations for Anomaly Detection in Time Series
In this work we propose a model-agnostic algorithm that generates counterfactual ensemble explanations for time series anomaly detection models.
Memory Efficient Continual Learning with Transformers
Moreover, applications increasingly rely on large pre-trained neural networks, such as pre-trained Transformers, since the resources or data might not be available in sufficiently large quantities to practitioners to train the model from scratch.
Meta-Forecasting by combining Global Deep Representations with Local Adaptation
While classical time series forecasting considers individual time series in isolation, recent advances based on deep learning showed that jointly learning from a large pool of related time series can boost the forecasting accuracy.
A multi-objective perspective on jointly tuning hardware and hyperparameters
In three benchmarks where hardware is selected in addition to hyperparameters, we obtain runtime and cost reductions of at least 5. 8x and 8. 8x, respectively.
Dynamic Pruning of a Neural Network via Gradient Signal-to-Noise Ratio
Dynamic sparsity pruning undoes this limitation and allows to adapt the structure of the sparse neural network during training.
Automatic Termination for Hyperparameter Optimization
Across an extensive range of real-world HPO problems and baselines, we show that our termination criterion achieves a better trade-off between the test performance and optimization time.
BORE: Bayesian Optimization by Density-Ratio Estimation
Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods.
Model-based Asynchronous Hyperparameter and Neural Architecture Search
We introduce a model-based asynchronous multi-fidelity method for hyperparameter and neural architecture search that combines the strengths of asynchronous Hyperband and Gaussian process-based Bayesian optimization.
Cost-aware Bayesian Optimization
Bayesian optimization (BO) is a class of global optimization algorithms, suitable for minimizing an expensive objective function in as few function evaluations as possible.
LEEP: A New Measure to Evaluate Transferability of Learned Representations
We introduce a new measure to evaluate the transferability of representations learned by classifiers.
Ranked #4 on
Transferability
on classification benchmark
Constrained Bayesian Optimization with Max-Value Entropy Search
We propose constrained Max-value Entropy Search (cMES), a novel information theoretic-based acquisition function implementing this formulation.
Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-box optimization problems.
Scalable Hyperparameter Transfer Learning
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization.
Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization.
Bayesian Optimization with Tree-structured Dependencies
The benefit of leveraging this structure is twofold: we explore the search space more efficiently and posterior inference scales more favorably with the number of observations than Gaussian Process-based approaches published in the literature.
Online optimization and regret guarantees for non-additive long-term constraints
We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game.
Incremental Variational Inference for Latent Dirichlet Allocation
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA).
Overlapping Trace Norms in Multi-View Learning
Multi-view learning leverages correlations between different sources of data to make predictions in one view based on observations in another view.
