Search Results for author: Martin Wistuba

Found 30 papers, 8 papers with code

Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need

no code implementations5 Jun 2024 Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, Giovanni Zappella

In this paper, we conduct this research and find that the choice of prompt tuning as a PEFT method hurts the overall performance of the CL system.

Continual Learning

Continual Learning with Low Rank Adaptation

no code implementations29 Nov 2023 Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, Giovanni Zappella

Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest.

Continual Learning Incremental Learning

Renate: A Library for Real-World Continual Learning

1 code implementation24 Apr 2023 Martin Wistuba, Martin Ferianc, Lukas Balles, Cedric Archambeau, Giovanni Zappella

We discuss requirements for the use of continual learning algorithms in practice, from which we derive design principles for Renate.

Continual Learning

Variational Boosted Soft Trees

no code implementations21 Feb 2023 Tristan Cinquin, Tammo Rukat, Philipp Schmidt, Martin Wistuba, Artur Bekasov

Variational inference is often used to implement Bayesian neural networks, but is difficult to apply to GBMs, because the decision trees used as weak learners are non-differentiable.

Decision Making Out-of-Distribution Detection +3

PASHA: Efficient HPO and NAS with Progressive Resource Allocation

2 code implementations14 Jul 2022 Ondrej Bohdal, Lukas Balles, Martin Wistuba, Beyza Ermis, Cédric Archambeau, Giovanni Zappella

Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in practice they can be costly to run.

BIG-bench Machine Learning Hyperparameter Optimization +1

Continual Learning with Transformers for Image Classification

no code implementations28 Jun 2022 Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, Cedric Archambeau

This phenomenon is known as catastrophic forgetting and it is often difficult to prevent due to practical constraints, such as the amount of data that can be stored or the limited computation sources that can be used.

Continual Learning Image Classification +2

Memory Efficient Continual Learning with Transformers

no code implementations9 Mar 2022 Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, Cedric Archambeau

Moreover, applications increasingly rely on large pre-trained neural networks, such as pre-trained Transformers, since the resources or data might not be available in sufficiently large quantities to practitioners to train the model from scratch.

Continual Learning text-classification +1

Supervising the Multi-Fidelity Race of Hyperparameter Configurations

1 code implementation20 Feb 2022 Martin Wistuba, Arlind Kadra, Josif Grabocka

Multi-fidelity (gray-box) hyperparameter optimization techniques (HPO) have recently emerged as a promising direction for tuning Deep Learning methods.

Bayesian Optimization Gaussian Processes +1

HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML

1 code implementation11 Jun 2021 Sebastian Pineda Arango, Hadi S. Jomaa, Martin Wistuba, Josif Grabocka

Hyperparameter optimization (HPO) is a core problem for the machine learning community and remains largely unsolved due to the significant computational resources required to evaluate hyperparameter configurations.

Hyperparameter Optimization Transfer Learning

A Comprehensive Survey on Hardware-Aware Neural Architecture Search

no code implementations22 Jan 2021 Hadjer Benmeziane, Kaoutar El Maghraoui, Hamza Ouarnoughi, Smail Niar, Martin Wistuba, Naigang Wang

Arguably their most significant impact has been in image classification and object detection tasks where the state of the art results have been obtained.

Hardware Aware Neural Architecture Search Image Classification +3

Few-Shot Bayesian Optimization with Deep Kernel Surrogates

1 code implementation ICLR 2021 Martin Wistuba, Josif Grabocka

Hyperparameter optimization (HPO) is a central pillar in the automation of machine learning solutions and is mainly performed via Bayesian optimization, where a parametric surrogate is learned to approximate the black box response function (e. g. validation error).

Bayesian Optimization Few-Shot Learning +2

Learning to Rank Learning Curves

no code implementations ICML 2020 Martin Wistuba, Tejaswini Pedapati

Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many different model configurations.

Learning-To-Rank Neural Architecture Search +1

XferNAS: Transfer Neural Architecture Search

no code implementations18 Jul 2019 Martin Wistuba

In experiments on CIFAR-10 and CIFAR-100, we observe a reduction in the search time from 200 to only 6 GPU days, a speed up by a factor of 33.

Neural Architecture Search

A Survey on Neural Architecture Search

no code implementations4 May 2019 Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati

The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search.

Data Augmentation Evolutionary Algorithms +1

Inductive Transfer for Neural Architecture Optimization

no code implementations8 Mar 2019 Martin Wistuba, Tejaswini Pedapati

First, we propose a novel neural architecture selection method which employs this knowledge to identify strong and weak characteristics of neural architectures across datasets.

Image Classification Neural Architecture Search

Adversarial Robustness Toolbox v1.0.0

6 code implementations3 Jul 2018 Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian M. Molloy, Ben Edwards

Defending Machine Learning models involves certifying and verifying model robustness and model hardening with approaches such as pre-processing inputs, augmenting training data with adversarial samples, and leveraging runtime detection methods to flag any inputs that might have been modified by an adversary.

Adversarial Robustness BIG-bench Machine Learning +2

Automated Image Data Preprocessing with Deep Reinforcement Learning

1 code implementation15 Jun 2018 Tran Ngoc Minh, Mathieu Sinn, Hoang Thanh Lam, Martin Wistuba

Data preparation, i. e. the process of transforming raw data into a format that can be used for training effective machine learning models, is a tedious and time-consuming task.

reinforcement-learning Reinforcement Learning (RL)

Scalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data

no code implementations7 Jun 2018 Martin Wistuba, Ambrish Rawat

We introduce a new Bayesian multi-class support vector machine by formulating a pseudo-likelihood for a multi-class hinge loss in the form of a location-scale mixture of Gaussians.

Active Learning General Classification +1

Neural Feature Learning From Relational Database

no code implementations16 Jan 2018 Hoang Thanh Lam, Tran Ngoc Minh, Mathieu Sinn, Beat Buesser, Martin Wistuba

To the best of our knowledge, this is the first time an automated data science system could win medals in Kaggle competitions with complex relational database.

Feature Engineering

Finding Competitive Network Architectures Within a Day Using UCT

no code implementations20 Dec 2017 Martin Wistuba

We adapt the UCT algorithm to the needs of network architecture search by proposing two ways of sharing information between different branches of the search tree.

Neural Architecture Search

Adversarial Phenomenon in the Eyes of Bayesian Deep Learning

no code implementations22 Nov 2017 Ambrish Rawat, Martin Wistuba, Maria-Irina Nicolae

Deep Learning models are vulnerable to adversarial examples, i. e.\ images obtained via deliberate imperceptible perturbations, such that the model misclassifies them with high confidence.

Automatic Frankensteining: Creating Complex Ensembles Autonomously

no code implementations SIAM 2017 2017 Martin Wistuba, Nicolas Schilling, Lars Schmidt-Thieme

Automating machine learning by providing techniques that autonomously find the best algorithm, hyperparameter configuration and preprocessing is helpful for both researchers and practitioners.

AutoML BIG-bench Machine Learning

Bank Card Usage Prediction Exploiting Geolocation Information

no code implementations13 Oct 2016 Martin Wistuba, Nghia Duong-Trung, Nicolas Schilling, Lars Schmidt-Thieme

We describe the solution of team ISMLL for the ECML-PKDD 2016 Discovery Challenge on Bank Card Usage for both tasks.

General Classification Position +1

Ultra-Fast Shapelets for Time Series Classification

no code implementations17 Mar 2015 Martin Wistuba, Josif Grabocka, Lars Schmidt-Thieme

A method for using shapelets for multivariate time series is proposed and Ultra-Fast Shapelets is proven to be successful in comparison to state-of-the-art multivariate time series classifiers on 15 multivariate time series datasets from various domains.

Classification General Classification +3

Scalable Discovery of Time-Series Shapelets

no code implementations11 Mar 2015 Josif Grabocka, Martin Wistuba, Lars Schmidt-Thieme

Time-series classification is an important problem for the data mining community due to the wide range of application domains involving time-series data.

Clustering General Classification +4

Time-Series Classification Through Histograms of Symbolic Polynomials

no code implementations24 Jul 2013 Josif Grabocka, Martin Wistuba, Lars Schmidt-Thieme

The coefficients of the polynomial functions are converted to symbolic words via equivolume discretizations of the coefficients' distributions.

Classification Econometrics +4

Cannot find the paper you are looking for? You can Submit a new open access paper.