Search Results for author:
Found 37 papers, 19 papers with code
Multi-objective Differentiable Neural Architecture Search
Pareto front profiling in multi-objective optimization (MOO), i. e. finding a diverse set of Pareto optimal solutions, is challenging, especially with expensive objectives like neural network training.
Hierarchical Transformers are Efficient Meta-Reinforcement Learners
We introduce Hierarchical Transformers for Meta-Reinforcement Learning (HTrMRL), a powerful online meta-reinforcement learning approach.
Tabular Data: Is Attention All You Need?
In this paper, we introduce a large-scale empirical study comparing neural networks against gradient-boosted decision trees on tabular data, but also transformer-based architectures against traditional multi-layer perceptrons (MLP) with residual connections.
Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How
With the ever-increasing number of pretrained models, machine learning practitioners are continuously faced with which pretrained model to use, and how to finetune it for a new dataset.
Deep Pipeline Embeddings for AutoML
As a remedy, this paper proposes a novel neural architecture that captures the deep interaction between the components of a Machine Learning pipeline.
Automatic Machine Learning Model Selection
Bayesian Optimization
+2
Breaking the Paradox of Explainable Deep Learning
Through extensive experiments, we demonstrate that our explainable deep networks are as accurate as state-of-the-art classifiers on tabular data.
Phantom Embeddings: Using Embedding Space for Model Regularization in Deep Neural Networks
We create phantom embeddings from a subset of homogenous samples and use these phantom embeddings to decrease the inter-class similarity of instances in their latent embedding space.
Deep Ranking Ensembles for Hyperparameter Optimization
Automatically optimizing the hyperparameters of Machine Learning algorithms is one of the primary open questions in AI.
Zero-Shot AutoML with Pretrained Models
Given a new dataset D and a low compute budget, how should we choose a pre-trained model to fine-tune to D, and set the fine-tuning hyperparameters without risking overfitting, particularly if D is small?
Supervising the Multi-Fidelity Race of Hyperparameter Configurations
Multi-fidelity (gray-box) hyperparameter optimization techniques (HPO) have recently emerged as a promising direction for tuning Deep Learning methods.
Transformers Can Do Bayesian Inference
Our method restates the objective of posterior approximation as a supervised classification problem with a set-valued input: it repeatedly draws a task (or function) from the prior, draws a set of data points and their labels from it, masks one of the labels and learns to make probabilistic predictions for it based on the set-valued input of the rest of the data points.
Multi-task problems are not multi-objective
These works also use Multi-Task Learning (MTL) problems to benchmark MOO algorithms treating each task as independent objective.
Transfer Learning for Bayesian HPO with End-to-End Meta-Features
As a result, our novel DKLM can learn contextualized dataset-specific similarity representations for hyperparameter configurations.
Well-tuned Simple Nets Excel on Tabular Datasets
Tabular datasets are the last "unconquered castle" for deep learning, with traditional ML methods like Gradient-Boosted Decision Trees still performing strongly even against recent specialized neural architectures.
HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
Hyperparameter optimization (HPO) is a core problem for the machine learning community and remains largely unsolved due to the significant computational resources required to evaluate hyperparameter configurations.
Scalable Pareto Front Approximation for Deep Multi-Objective Learning
Prior work either demand optimizing a new network for every point on the Pareto front, or induce a large overhead to the number of trainable parameters by using hyper-networks conditioned on modifiable preferences.
Hyperparameter Optimization with Differentiable Metafeatures
In contrast to existing models, DMFBS i) integrates a differentiable metafeature extractor and ii) is optimized using a novel multi-task loss, linking manifold regularization with a dataset similarity measure learned via an auxiliary dataset identification meta-task, effectively enforcing the response approximation for similar datasets to be similar.
Few-Shot Bayesian Optimization with Deep Kernel Surrogates
Hyperparameter optimization (HPO) is a central pillar in the automation of machine learning solutions and is mainly performed via Bayesian optimization, where a parametric surrogate is learned to approximate the black box response function (e. g. validation error).
Zero-shot Transfer Learning for Gray-box Hyper-parameter Optimization
Zero-shot hyper-parameter optimization refers to the process of selecting hyper- parameter configurations that are expected to perform well for a given dataset upfront, without access to any observations of the losses of the target response.
Regularization Cocktails
The regularization of prediction models is arguably the most crucial ingredient that allows Machine Learning solutions to generalize well on unseen data.
NASLib: A Modular and Flexible Neural Architecture Search Library
Neural Architecture Search (NAS) is one of the focal points for the Deep Learning community, but reproducing NAS methods is extremely challenging due to numerous low-level implementation details.
HIDRA: Head Initialization across Dynamic targets for Robust Architectures
In this paper, we present HIDRA, a meta-learning approach that enables training and evaluating across tasks with any number of target variables.
Chameleon: Learning Model Initializations Across Tasks With Different Schemas
Parametric models, and particularly neural networks, require weight initialization as a starting point for gradient-based optimization.
Atomic Compression Networks
Compressed forms of deep neural networks are essential in deploying large-scale computational models on resource-constrained devices.
Hyp-RL : Hyperparameter Optimization by Reinforcement Learning
More recently, methods have been introduced that build a so-called surrogate model that predicts the validation loss for a specific hyperparameter setting, model and dataset and then sequentially select the next hyperparameter to test, based on a heuristic function of the expected value and the uncertainty of the surrogate model called acquisition function (sequential model-based Bayesian optimization, SMBO).
In Hindsight: A Smooth Reward for Steady Exploration
In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy.
Dataset2Vec: Learning Dataset Meta-Features
As a data-driven approach, meta-learning requires meta-features that represent the primary learning tasks or datasets, and are estimated traditonally as engineered dataset statistics that require expert domain knowledge tailored for every meta-task.
Learning Surrogate Losses
Ultimately, the surrogate losses are learned jointly with the prediction model via bilevel optimization.
Multi-Label Network Classification via Weighted Personalized Factorizations
It can be formalized as a multi-relational learning task for predicting nodes labels based on their relations within the network.
Data-Driven Vehicle Trajectory Forecasting
An active area of research is to increase the safety of self-driving vehicles.
NeuralWarp: Time-Series Similarity with Warping Networks
Research on time-series similarity measures has emphasized the need for elastic methods which align the indices of pairs of time series and a plethora of non-parametric have been proposed for the task.
Channel masking for multivariate time series shapelets
Time series shapelets are discriminative sub-sequences and their similarity to time series can be used for time series classification.
Optimal Time-Series Motifs
We demonstrate that searching is non-optimal since the domain of motifs is restricted, and instead we propose a principled optimization approach able to find optimal motifs.
Ultra-Fast Shapelets for Time Series Classification
A method for using shapelets for multivariate time series is proposed and Ultra-Fast Shapelets is proven to be successful in comparison to state-of-the-art multivariate time series classifiers on 15 multivariate time series datasets from various domains.
Scalable Discovery of Time-Series Shapelets
Time-series classification is an important problem for the data mining community due to the wide range of application domains involving time-series data.
Invariant Factorization Of Time-Series
Time-series classification is an important domain of machine learning and a plethora of methods have been developed for the task.
Time-Series Classification Through Histograms of Symbolic Polynomials
The coefficients of the polynomial functions are converted to symbolic words via equivolume discretizations of the coefficients' distributions.
