Search Results for author: Joaquin Vanschoren

Found 67 papers, 35 papers with code

Evolving Machine Learning: A Survey

no code implementations23 May 2025 Ignacio Cabrera Martin, Subhaditya Mukherjee, Almas Baimagambetov, Joaquin Vanschoren, Nikolaos Polatidis

Evolving Machine Learning (EML) has emerged as a critical paradigm, enabling continuous learning and adaptation in real-time data streams.

Meta-Learning Survey

CrypticBio: A Large Multimodal Dataset for Visually Confusing Biodiversity

1 code implementation16 May 2025 Georgiana Manolache, Gerard Schouten, Joaquin Vanschoren

To highlight the importance of the dataset, we benchmark a suite of state-of-the-art foundation models across CrypticBio subsets of common, unseen, endangered, and invasive species, and demonstrate the substantial impact of geographical context on vision-language zero-shot learning for cryptic species.

Zero-Shot Learning

AutoML Benchmark with shorter time constraints and early stopping

no code implementations1 Apr 2025 Israel Campero Jurado, Pieter Gijsbers, Joaquin Vanschoren

Automated Machine Learning (AutoML) automatically builds machine learning (ML) models on data.

AutoML

Sculpting [CLS] Features for Pre-Trained Model-Based Class-Incremental Learning

no code implementations20 Feb 2025 Murat Onur Yildirim, Elif Ceren Gok Yildirim, Joaquin Vanschoren

Excessive plasticity in the models breaks generalizability and causes forgetting, while strong stability results in insufficient adaptation to new classes.

class-incremental learning Class Incremental Learning +3

AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons

no code implementations19 Feb 2025 Shaona Ghosh, Heather Frase, Adina Williams, Sarah Luger, Paul Röttger, Fazl Barez, Sean McGregor, Kenneth Fricklas, Mala Kumar, Quentin Feuillade--Montixi, Kurt Bollacker, Felix Friedrich, Ryan Tsang, Bertie Vidgen, Alicia Parrish, Chris Knotz, Eleonora Presani, Jonathan Bennion, Marisa Ferrara Boston, Mike Kuniavsky, Wiebke Hutiri, James Ezick, Malek Ben Salem, Rajat Sahay, Sujata Goswami, Usman Gohar, Ben Huang, Supheakmungkol Sarin, Elie Alhajjar, Canyu Chen, Roman Eng, Kashyap Ramanandula Manjusha, Virendra Mehta, Eileen Long, Murali Emani, Natan Vidra, Benjamin Rukundo, Abolfazl Shahbazi, Kongtao Chen, Rajat Ghosh, Vithursan Thangarasa, Pierre Peigné, Abhinav Singh, Max Bartolo, Satyapriya Krishna, Mubashara Akhtar, Rafael Gold, Cody Coleman, Luis Oala, Vassil Tashev, Joseph Marvin Imperial, Amy Russ, Sasidhar Kunapuli, Nicolas Miailhe, Julien Delaunay, Bhaktipriya Radharapu, Rajat Shinde, Tuesday, Debojyoti Dutta, Declan Grabb, Ananya Gangavarapu, Saurav Sahay, Agasthya Gangavarapu, Patrick Schramowski, Stephen Singam, Tom David, Xudong Han, Priyanka Mary Mammen, Tarunima Prabhakar, Venelin Kovatchev, Rebecca Weiss, Ahmed Ahmed, Kelvin N. Manyeki, Sandeep Madireddy, Foutse khomh, Fedor Zhdanov, Joachim Baumann, Nina Vasan, Xianjun Yang, Carlos Mougn, Jibin Rajan Varghese, Hussain Chinoy, Seshakrishna Jitendar, Manil Maskey, Claire V. Hardgrove, TianHao Li, Aakash Gupta, Emil Joswin, Yifan Mai, Shachi H Kumar, Cigdem Patlak, Kevin Lu, Vincent Alessi, Sree Bhargavi Balija, Chenhe Gu, Robert Sullivan, James Gealy, Matt Lavrisa, James Goel, Peter Mattson, Percy Liang, Joaquin Vanschoren

This work represents a crucial step toward establishing global standards for AI risk and reliability evaluation while acknowledging the need for continued development in areas such as multiturn interactions, multimodal understanding, coverage of additional languages, and emerging hazard categories.

On dataset transferability in medical image classification

1 code implementation28 Dec 2024 Dovile Juodelyte, Enzo Ferrante, Yucheng Lu, Prabhant Singh, Joaquin Vanschoren, Veronika Cheplygina

These methods primarily focus on estimating the suitability of pre-trained source model features for a target dataset, which can lead to unrealistic predictions, such as suggesting that the target dataset is the best source for itself.

Benchmarking image-classification +2

Learning to Learn without Forgetting using Attention

1 code implementation6 Aug 2024 Anna Vettoruzzo, Joaquin Vanschoren, Mohamed-Rafik Bouguelia, Thorsteinn Rögnvaldsson

Continual learning (CL) refers to the ability to continually learn over time by accommodating new knowledge while retaining previously learned experience.

Continual Learning Meta-Learning

Can time series forecasting be automated? A benchmark and analysis

no code implementations23 Jul 2024 Anvitha Thirthapura Sreedhara, Joaquin Vanschoren

In the field of machine learning and artificial intelligence, time series forecasting plays a pivotal role across various domains such as finance, healthcare, and weather.

Benchmarking Decision Making +3

HyTAS: A Hyperspectral Image Transformer Architecture Search Benchmark and Analysis

1 code implementation23 Jul 2024 Fangqin Zhou, Mert Kilickaya, Joaquin Vanschoren, Ran Piao

Hyperspectral Imaging (HSI) plays an increasingly critical role in precise vision tasks within remote sensing, capturing a wide spectrum of visual data.

Model Discovery

CLAMS: A System for Zero-Shot Model Selection for Clustering

no code implementations15 Jul 2024 Prabhant Singh, Pieter Gijsbers, Murat Onur Yildirim, Elif Ceren Gok, Joaquin Vanschoren

We propose an AutoML system that enables model selection on clustering problems by leveraging optimal transport-based dataset similarity.

AutoML Clustering +1

Unsupervised Meta-Learning via In-Context Learning

no code implementations25 May 2024 Anna Vettoruzzo, Lorenzo Braccaioli, Joaquin Vanschoren, Marlena Nowaczyk

Unsupervised meta-learning aims to learn feature representations from unsupervised datasets that can transfer to downstream tasks with limited labeled data.

In-Context Learning Memorization +1

Introducing v0.5 of the AI Safety Benchmark from MLCommons

1 code implementation18 Apr 2024 Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller, Ram Gandikota, Agasthya Gangavarapu, Ananya Gangavarapu, James Gealy, Rajat Ghosh, James Goel, Usman Gohar, Sujata Goswami, Scott A. Hale, Wiebke Hutiri, Joseph Marvin Imperial, Surgan Jandial, Nick Judd, Felix Juefei-Xu, Foutse khomh, Bhavya Kailkhura, Hannah Rose Kirk, Kevin Klyman, Chris Knotz, Michael Kuchnik, Shachi H. Kumar, Srijan Kumar, Chris Lengerich, Bo Li, Zeyi Liao, Eileen Peters Long, Victor Lu, Sarah Luger, Yifan Mai, Priyanka Mary Mammen, Kelvin Manyeki, Sean McGregor, Virendra Mehta, Shafee Mohammed, Emanuel Moss, Lama Nachman, Dinesh Jinenhally Naganna, Amin Nikanjam, Besmira Nushi, Luis Oala, Iftach Orr, Alicia Parrish, Cigdem Patlak, William Pietri, Forough Poursabzi-Sangdeh, Eleonora Presani, Fabrizio Puletti, Paul Röttger, Saurav Sahay, Tim Santos, Nino Scherrer, Alice Schoenauer Sebag, Patrick Schramowski, Abolfazl Shahbazi, Vin Sharma, Xudong Shen, Vamsi Sistla, Leonard Tang, Davide Testuggine, Vithursan Thangarasa, Elizabeth Anne Watkins, Rebecca Weiss, Chris Welty, Tyler Wilbers, Adina Williams, Carole-Jean Wu, Poonam Yadav, Xianjun Yang, Yi Zeng, Wenhui Zhang, Fedor Zhdanov, Jiacheng Zhu, Percy Liang, Peter Mattson, Joaquin Vanschoren

We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.

FOCIL: Finetune-and-Freeze for Online Class Incremental Learning by Training Randomly Pruned Sparse Experts

1 code implementation13 Mar 2024 Murat Onur Yildirim, Elif Ceren Gok Yildirim, Decebal Constantin Mocanu, Joaquin Vanschoren

Class incremental learning (CIL) in an online continual learning setting strives to acquire knowledge on a series of novel classes from a data stream, using each data point only once for training.

class-incremental learning Class Incremental Learning +1

Automatic Combination of Sample Selection Strategies for Few-Shot Learning

no code implementations5 Feb 2024 Branislav Pecher, Ivan Srba, Maria Bielikova, Joaquin Vanschoren

In few-shot learning, such as meta-learning, few-shot fine-tuning or in-context learning, the limited number of samples used to train a model have a significant impact on the overall success.

Few-Shot Learning In-Context Learning

What Can AutoML Do For Continual Learning?

no code implementations20 Nov 2023 Mert Kilickaya, Joaquin Vanschoren

This position paper outlines the potential of AutoML for incremental (continual) learning to encourage more research in this direction.

AutoML Continual Learning +1

Locality-Aware Hyperspectral Classification

1 code implementation4 Sep 2023 Fangqin Zhou, Mert Kilickaya, Joaquin Vanschoren

Hyperspectral image classification is gaining popularity for high-precision vision tasks in remote sensing, thanks to their ability to capture visual information available in a wide continuum of spectra.

 Ranked #1 on Hyperspectral Image Classification on Houston (OA@15perclass metric)

Classification Hyperspectral Image Classification +1

Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates

1 code implementation28 Aug 2023 Murat Onur Yildirim, Elif Ceren Gok Yildirim, Ghada Sokar, Decebal Constantin Mocanu, Joaquin Vanschoren

Therefore, we perform a comprehensive study in which we investigate various DST components to find the best topology per task on well-known CIFAR100 and miniImageNet benchmarks in a task-incremental CL setup since our primary focus is to evaluate the performance of various DST criteria, rather than the process of mask selection.

Continual Learning

Advances and Challenges in Meta-Learning: A Technical Review

no code implementations10 Jul 2023 Anna Vettoruzzo, Mohamed-Rafik Bouguelia, Joaquin Vanschoren, Thorsteinn Rögnvaldsson, KC Santosh

This review provides a comprehensive technical overview of meta-learning, emphasizing its importance in real-world applications where data may be scarce or expensive to obtain.

Continual Learning Domain Adaptation +4

Neural Architecture Search for Visual Anomaly Segmentation

1 code implementation18 Apr 2023 Tommie Kerssies, Joaquin Vanschoren

This paper presents the first application of neural architecture search to the complex task of segmenting visual anomalies.

Anomaly Segmentation Neural Architecture Search

AdaCL:Adaptive Continual Learning

1 code implementation23 Mar 2023 Elif Ceren Gok Yildirim, Murat Onur Yildirim, Mert Kilickaya, Joaquin Vanschoren

We show that adapting hyperpararmeters on each new task leads to improvement in accuracy, forgetting and memory.

Bayesian Optimization class-incremental learning +2

Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML

no code implementations15 Mar 2023 Hilde Weerts, Florian Pfisterer, Matthias Feurer, Katharina Eggensperger, Edward Bergman, Noor Awad, Joaquin Vanschoren, Mykola Pechenizkiy, Bernd Bischl, Frank Hutter

The field of automated machine learning (AutoML) introduces techniques that automate parts of the development of machine learning (ML) systems, accelerating the process and reducing barriers for novices.

AutoML Fairness

Are Labels Needed for Incremental Instance Learning?

no code implementations26 Jan 2023 Mert Kilickaya, Joaquin Vanschoren

We propose VINIL, a self-incremental learner that can learn object instances sequentially, ii.

Object

Automated Imbalanced Learning

1 code implementation1 Nov 2022 Prabhant Singh, Joaquin Vanschoren

Automated Machine Learning has grown very successful in automating the time-consuming, iterative tasks of machine learning model development.

AutoML

Meta-Learning for Unsupervised Outlier Detection with Optimal Transport

no code implementations1 Nov 2022 Prabhant Singh, Joaquin Vanschoren

Automated machine learning has been widely researched and adopted in the field of supervised classification and regression, but progress in unsupervised settings has been limited.

Meta-Learning Outlier Detection

Evaluating Continual Test-Time Adaptation for Contextual and Semantic Domain Shifts

1 code implementation18 Aug 2022 Tommie Kerssies, Mert Kılıçkaya, Joaquin Vanschoren

In this paper, our goal is to adapt a pre-trained convolutional neural network to domain shifts at test time.

Test-time Adaptation

AMLB: an AutoML Benchmark

2 code implementations25 Jul 2022 Pieter Gijsbers, Marcos L. P. Bueno, Stefan Coors, Erin LeDell, Sébastien Poirier, Janek Thomas, Bernd Bischl, Joaquin Vanschoren

Comparing different AutoML frameworks is notoriously challenging and often done incorrectly.

AutoML

Open-Ended Learning Strategies for Learning Complex Locomotion Skills

no code implementations14 Jun 2022 Fangqin Zhou, Joaquin Vanschoren

Teaching robots to learn diverse locomotion skills under complex three-dimensional environmental settings via Reinforcement Learning (RL) is still challenging.

Diversity Reinforcement Learning (RL)

Warm-starting DARTS using meta-learning

no code implementations12 May 2022 Matej Grobelnik, Joaquin Vanschoren

Neural architecture search (NAS) has shown great promise in the field of automated machine learning (AutoML).

Meta-Learning Neural Architecture Search

Advances in MetaDL: AAAI 2021 challenge and workshop

no code implementations1 Feb 2022 Adrian El Baz, Isabelle Guyon, Zhengying Liu, Jan van Rijn, Sebastien Treguer, Joaquin Vanschoren

Winning methods featured various classifiers trained on top of the second last layer of popular CNN backbones, fined-tuned on the meta-training data (not necessarily in an episodic manner), then trained on the labeled support and tested on the unlabeled query sets of the meta-test data.

Few-Shot Learning

Online AutoML: An adaptive AutoML framework for online learning

no code implementations24 Jan 2022 Bilge Celik, Prabhant Singh, Joaquin Vanschoren

For this purpose, we design an adaptive Online Automated Machine Learning (OAML) system, searching the complete pipeline configuration space of online learners, including preprocessing algorithms and ensembling techniques.

AutoML BIG-bench Machine Learning

Automated Reinforcement Learning: An Overview

no code implementations13 Jan 2022 Reza Refaei Afshar, Yingqian Zhang, Joaquin Vanschoren, Uzay Kaymak

Automated RL provides a framework in which different components of RL including MDP modeling, algorithm selection and hyper-parameter optimization are modeled and defined automatically.

Decision Making Deep Reinforcement Learning +3

Frugal Machine Learning

no code implementations5 Nov 2021 Mikhail Evchenko, Joaquin Vanschoren, Holger H. Hoos, Marc Schoenauer, Michèle Sebag

Machine learning, already at the core of increasingly many systems and applications, is set to become even more ubiquitous with the rapid rise of wearable devices and the Internet of Things.

Activity Recognition BIG-bench Machine Learning

From Strings to Data Science: a Practical Framework for Automated String Handling

no code implementations2 Nov 2021 John W. van Lith, Joaquin Vanschoren

Many machine learning libraries require that string features be converted to a numerical representation for the models to work as intended.

Cats, not CAT scans: a study of dataset similarity in transfer learning for 2D medical image classification

1 code implementation13 Jul 2021 Irma van den Brandt, Floris Fok, Bas Mulders, Joaquin Vanschoren, Veronika Cheplygina

There is currently no consensus on how to choose appropriate source data, and in the literature we can find both evidence of favoring large natural image datasets such as ImageNet, and evidence of favoring more specialized medical datasets.

image-classification Image Classification +2

Meta-Learning for Symbolic Hyperparameter Defaults

1 code implementation10 Jun 2021 Pieter Gijsbers, Florian Pfisterer, Jan N. van Rijn, Bernd Bischl, Joaquin Vanschoren

Hyperparameter optimization in machine learning (ML) deals with the problem of empirically learning an optimal algorithm configuration from data, usually formulated as a black-box optimization problem.

Hyperparameter Optimization Meta-Learning

Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms

1 code implementation3 Feb 2021 Rishabh Goyal, Joaquin Vanschoren, Victor van Acht, Stephan Nijssen

One drawback however is the high computational complexity and high memory consumption of CNNs which makes them unfeasible for execution on embedded platforms which are constrained on physical resources needed to support CNNs.

image-classification Image Classification +1

Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models

no code implementations6 Jan 2021 Jeroen van Hoof, Joaquin Vanschoren

Bayesian Optimization is a popular tool for tuning algorithms in automatic machine learning (AutoML) systems.

Bayesian Optimization Gaussian Processes +2

Theory-based Habit Modeling for Enhancing Behavior Prediction

no code implementations5 Jan 2021 Chao Zhang, Joaquin Vanschoren, Arlette van Wissen, Daniel Lakens, Boris de Ruyter, Wijnand A. IJsselsteijn

Psychological theories of habit posit that when a strong habit is formed through behavioral repetition, it can trigger behavior automatically in the same environment.

Prediction

Aerial Imagery Pixel-level Segmentation

1 code implementation3 Dec 2020 Michael R. Heffels, Joaquin Vanschoren

Hence, we also propose a new benchmark on the DroneDeploy test set using the best performing DeepLabv3+ Xception65 architecture, with a mIOU score of 52. 5%.

Data Augmentation Segmentation +1

GAMA: a General Automated Machine learning Assistant

3 code implementations9 Jul 2020 Pieter Gijsbers, Joaquin Vanschoren

The General Automated Machine learning Assistant (GAMA) is a modular AutoML system developed to empower users to track and control how AutoML algorithms search for optimal machine learning pipelines, and facilitate AutoML research itself.

AutoML Benchmarking +1

Adaptation Strategies for Automated Machine Learning on Evolving Data

1 code implementation9 Jun 2020 Bilge Celik, Joaquin Vanschoren

To that end, we propose 6 concept drift adaptation strategies and evaluate their effectiveness on different AutoML approaches.

AutoML Bayesian Optimization +1

OpenML-Python: an extensible Python API for OpenML

1 code implementation6 Nov 2019 Matthias Feurer, Jan N. van Rijn, Arlind Kadra, Pieter Gijsbers, Neeratyoy Mallik, Sahithya Ravi, Andreas Müller, Joaquin Vanschoren, Frank Hutter

It also provides functionality to conduct machine learning experiments, upload the results to OpenML, and reproduce results which are stored on OpenML.

BIG-bench Machine Learning

An Open Source AutoML Benchmark

no code implementations1 Jul 2019 Pieter Gijsbers, Erin LeDell, Janek Thomas, Sébastien Poirier, Bernd Bischl, Joaquin Vanschoren

In recent years, an active field of research has developed around automated machine learning (AutoML).

AutoML BIG-bench Machine Learning

Transformative Machine Learning

no code implementations8 Nov 2018 Ivan Olier, Oghenejokpeme I. Orhobor, Joaquin Vanschoren, Ross D. King

In all three problems, transformative machine learning significantly outperforms the best intrinsic representation.

BIG-bench Machine Learning Drug Design +4

Meta-Learning: A Survey

no code implementations8 Oct 2018 Joaquin Vanschoren

In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.

BIG-bench Machine Learning Meta-Learning +1

Characterizing classification datasets: a study of meta-features for meta-learning

2 code implementations30 Aug 2018 Adriano Rivolli, Luís P. F. Garcia, Carlos Soares, Joaquin Vanschoren, André C. P. L. F. de Carvalho

These characterizations, also called meta-features, describe properties of the data which are predictive for the performance of machine learning algorithms trained on them.

BIG-bench Machine Learning General Classification +1

ML-Schema: Exposing the Semantics of Machine Learning with Schemas and Ontologies

no code implementations14 Jul 2018 Gustavo Correa Publio, Diego Esteves, Agnieszka Ławrynowicz, Panče Panov, Larisa Soldatova, Tommaso Soru, Joaquin Vanschoren, Hamid Zafar

The ML-Schema, proposed by the W3C Machine Learning Schema Community Group, is a top-level ontology that provides a set of classes, properties, and restrictions for representing and interchanging information on machine learning algorithms, datasets, and experiments.

BIG-bench Machine Learning

Meta-QSAR: a large-scale application of meta-learning to drug design and discovery

no code implementations12 Sep 2017 Ivan Olier, Noureddin Sadawi, G. Richard Bickerton, Joaquin Vanschoren, Crina Grosan, Larisa Soldatova, Ross D. King

We first carried out the most comprehensive ever comparison of machine learning methods for QSAR learning: 18 regression methods, 6 molecular representations, applied to more than 2, 700 QSAR problems.

BIG-bench Machine Learning Drug Design +2

ASlib: A Benchmark Library for Algorithm Selection

2 code implementations8 Jun 2015 Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Marius Lindauer, Yuri Malitsky, Alexandre Frechette, Holger Hoos, Frank Hutter, Kevin Leyton-Brown, Kevin Tierney, Joaquin Vanschoren

To address this problem, we introduce a standardized format for representing algorithm selection scenarios and a repository that contains a growing number of data sets from the literature.

OpenML: networked science in machine learning

1 code implementation29 Jul 2014 Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, Luis Torgo

Many sciences have made significant breakthroughs by adopting online tools that help organize, structure and mine information that is too detailed to be printed in journals.

BIG-bench Machine Learning

Open science in machine learning

no code implementations24 Feb 2014 Joaquin Vanschoren, Mikio L. Braun, Cheng Soon Ong

We present OpenML and mldata, open science platforms that provides easy access to machine learning data, software and results to encourage further study and application.

BIG-bench Machine Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.