Search Results for author: Bernd Bischl

Found 126 papers, 60 papers with code

On the Robustness of Global Feature Effect Explanations

no code implementations13 Jun 2024 Hubert Baniecki, Giuseppe Casalicchio, Bernd Bischl, Przemyslaw Biecek

We study the robustness of global post-hoc explanations for predictive models trained on tabular data.

A Large-Scale Neutral Comparison Study of Survival Models on Low-Dimensional Data

1 code implementation6 Jun 2024 Lukas Burk, John Zobolas, Bernd Bischl, Andreas Bender, Marvin N. Wright, Raphael Sonabend

Evaluating on 8 survival metrics, we assess discrimination, calibration, and overall predictive performance of the tested models.

Survival Analysis

Position: A Call to Action for a Human-Centered AutoML Paradigm

no code implementations5 Jun 2024 Marius Lindauer, Florian Karl, Anne Klier, Julia Moosbauer, Alexander Tornede, Andreas Mueller, Frank Hutter, Matthias Feurer, Bernd Bischl

Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience.

AutoML Position

FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models

no code implementations28 May 2024 Yang Zhang, Yawei Li, Xinpeng Wang, Qianli Shen, Barbara Plank, Bernd Bischl, Mina Rezaei, Kenji Kawaguchi

Overparametrized transformer networks are the state-of-the-art architecture for Large Language Models (LLMs).

Decoder

Position: Why We Must Rethink Empirical Research in Machine Learning

no code implementations3 May 2024 Moritz Herrmann, F. Julian D. Lange, Katharina Eggensperger, Giuseppe Casalicchio, Marcel Wever, Matthias Feurer, David Rügamer, Eyke Hüllermeier, Anne-Laure Boulesteix, Bernd Bischl

We warn against a common but incomplete understanding of empirical research in machine learning that leads to non-replicable results, makes findings unreliable, and threatens to undermine progress in the field.

Position

A Guide to Feature Importance Methods for Scientific Inference

1 code implementation19 Apr 2024 Fiona Katharina Ewald, Ludwig Bothmann, Marvin N. Wright, Bernd Bischl, Giuseppe Casalicchio, Gunnar König

Understanding the DGP requires insights into feature-target associations, which many ML models cannot directly provide, due to their opaque internal mechanisms.

Feature Importance

CountARFactuals -- Generating plausible model-agnostic counterfactual explanations with adversarial random forests

no code implementations4 Apr 2024 Susanne Dandl, Kristin Blesch, Timo Freiesleben, Gunnar König, Jan Kapar, Bernd Bischl, Marvin Wright

Counterfactual explanations elucidate algorithmic decisions by pointing to scenarios that would have led to an alternative, desired outcome.

counterfactual

Effector: A Python package for regional explanations

1 code implementation3 Apr 2024 Vasilis Gkolemis, Christos Diou, Eirini Ntoutsi, Theodore Dalamagas, Bernd Bischl, Julia Herbinger, Giuseppe Casalicchio

Effector implements well-established global effect methods, assesses the heterogeneity of each method and, based on that, provides regional effects.

Training Survival Models using Scoring Rules

no code implementations19 Mar 2024 Philipp Kopper, David Rügamer, Raphael Sonabend, Bernd Bischl, Andreas Bender

Survival Analysis provides critical insights for partially incomplete time-to-event data in various domains.

Survival Analysis

Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration

no code implementations7 Mar 2024 Julian Rodemann, Federico Croppi, Philipp Arens, Yusuf Sale, Julia Herbinger, Bernd Bischl, Eyke Hüllermeier, Thomas Augustin, Conor J. Walsh, Giuseppe Casalicchio

We address this issue by proposing ShapleyBO, a framework for interpreting BO's proposals by game-theoretic Shapley values. They quantify each parameter's contribution to BO's acquisition function.

Bayesian Optimization Gaussian Processes

Position Paper: Bridging the Gap Between Machine Learning and Sensitivity Analysis

no code implementations20 Dec 2023 Christian A. Scholbeck, Julia Moosbauer, Giuseppe Casalicchio, Hoshin Gupta, Bernd Bischl, Christian Heumann

We argue that interpretations of machine learning (ML) models or the model-building process can bee seen as a form of sensitivity analysis (SA), a general methodology used to explain complex systems in many fields such as environmental modeling, engineering, or economics.

Position

ConstraintMatch for Semi-constrained Clustering

1 code implementation26 Nov 2023 Jann Goschenhofer, Bernd Bischl, Zsolt Kira

Constrained clustering allows the training of classification models using pairwise constraints only, which are weak and relatively easy to mine, while still yielding full-supervision-level model performance.

Constrained Clustering

Post-hoc Orthogonalization for Mitigation of Protected Feature Bias in CXR Embeddings

1 code implementation2 Nov 2023 Tobias Weber, Michael Ingrisch, Bernd Bischl, David Rügamer

Methods: An orthogonalization is utilized to remove the influence of protected features (e. g., age, sex, race) in CXR embeddings, ensuring feature-independent results.

Image Classification

Evaluating machine learning models in non-standard settings: An overview and new findings

no code implementations23 Oct 2023 Roman Hornung, Malte Nalenz, Lennart Schneider, Andreas Bender, Ludwig Bothmann, Bernd Bischl, Thomas Augustin, Anne-Laure Boulesteix

Our findings corroborate the concern that standard resampling methods often yield biased GE estimates in non-standard settings, underscoring the importance of tailored GE estimation.

fmeffects: An R Package for Forward Marginal Effects

no code implementations3 Oct 2023 Holger Löwe, Christian A. Scholbeck, Christian Heumann, Bernd Bischl, Giuseppe Casalicchio

Forward marginal effects (FMEs) have recently been introduced as a versatile and effective model-agnostic interpretation method.

Probabilistic Self-supervised Learning via Scoring Rules Minimization

no code implementations5 Sep 2023 Amirhossein Vahidi, Simon Schoßer, Lisa Wimmer, Yawei Li, Bernd Bischl, Eyke Hüllermeier, Mina Rezaei

In this paper, we propose a novel probabilistic self-supervised learning via Scoring Rule Minimization (ProSMIN), which leverages the power of probabilistic models to enhance representation quality and mitigate collapsing representations.

Knowledge Distillation Out-of-Distribution Detection +2

Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning

no code implementations28 Aug 2023 Amirhossein Vahidi, Lisa Wimmer, Hüseyin Anil Gündüz, Bernd Bischl, Eyke Hüllermeier, Mina Rezaei

Ensembling a neural network is a widely recognized approach to enhance model performance, estimate uncertainty, and improve robustness in deep supervised learning.

Ensemble Learning Out-of-Distribution Detection +1

A Dual-Perspective Approach to Evaluating Feature Attribution Methods

no code implementations17 Aug 2023 Yawei Li, Yang Zhang, Kenji Kawaguchi, Ashkan Khakzar, Bernd Bischl, Mina Rezaei

We apply these metrics to mainstream attribution methods, offering a novel lens through which to analyze and compare feature attribution methods.

Q(D)O-ES: Population-based Quality (Diversity) Optimisation for Post Hoc Ensemble Selection in AutoML

no code implementations17 Jul 2023 Lennart Purucker, Lennart Schneider, Marie Anastacio, Joeran Beel, Bernd Bischl, Holger Hoos

Automated machine learning (AutoML) systems commonly ensemble models post hoc to improve predictive performance, typically via greedy ensemble selection (GES).

AutoML

Multi-Objective Optimization of Performance and Interpretability of Tabular Supervised Machine Learning Models

1 code implementation17 Jul 2023 Lennart Schneider, Bernd Bischl, Janek Thomas

Efficient optimization is achieved via augmentation of the search space of the learning algorithm by incorporating feature selection, interaction and monotonicity constraints into the hyperparameter search space.

feature selection Hyperparameter Optimization

How Different Is Stereotypical Bias Across Languages?

1 code implementation14 Jul 2023 Ibrahim Tolga Öztürk, Rostislav Nedelchev, Christian Heumann, Esteban Garces Arias, Marius Roger, Bernd Bischl, Matthias Aßenmacher

Recent studies have demonstrated how to assess the stereotypical bias in pre-trained English language models.

Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization

no code implementations7 Jul 2023 Chris Kolb, Christian L. Müller, Bernd Bischl, David Rügamer

Additionally, our theory establishes results of independent interest regarding matching local minima for arbitrary, potentially unregularized, objectives.

Sparse Learning

ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

1 code implementation16 Jun 2023 Lukas Rauch, Matthias Aßenmacher, Denis Huseljic, Moritz Wirth, Bernd Bischl, Bernhard Sick

Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most.

Active Learning

Decomposing Global Feature Effects Based on Feature Interactions

2 code implementations1 Jun 2023 Julia Herbinger, Bernd Bischl, Giuseppe Casalicchio

We formally introduce generalized additive decomposition of global effects (GADGET), which is a new framework based on recursive partitioning to find interpretable regions in the feature space such that the interaction-related heterogeneity of local feature effects is minimized.

Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learning

no code implementations25 May 2023 Daniel Saggau, Mina Rezaei, Bernd Bischl, Ilias Chalkidis

Learning quality document embeddings is a fundamental problem in natural language processing (NLP), information retrieval (IR), recommendation systems, and search engines.

Contrastive Learning Information Retrieval +5

Constrained Probabilistic Mask Learning for Task-specific Undersampled MRI Reconstruction

1 code implementation25 May 2023 Tobias Weber, Michael Ingrisch, Bernd Bischl, David Rügamer

Undersampling is a common method in Magnetic Resonance Imaging (MRI) to subsample the number of data points in k-space, reducing acquisition times at the cost of decreased image quality.

MRI Reconstruction

Deep Learning for Survival Analysis: A Review

1 code implementation24 May 2023 Simon Wiegrebe, Philipp Kopper, Raphael Sonabend, Bernd Bischl, Andreas Bender

The influx of deep learning (DL) techniques into the field of survival analysis in recent years has led to substantial methodological progress; for instance, learning from unstructured or high-dimensional data such as images, text or omics data.

Survival Analysis

Interpretable Regional Descriptors: Hyperbox-Based Local Explanations

no code implementations4 May 2023 Susanne Dandl, Giuseppe Casalicchio, Bernd Bischl, Ludwig Bothmann

This work introduces interpretable regional descriptors, or IRDs, for local, model-agnostic interpretations.

counterfactuals: An R Package for Counterfactual Explanation Methods

no code implementations13 Apr 2023 Susanne Dandl, Andreas Hofheinz, Martin Binder, Bernd Bischl, Giuseppe Casalicchio

Counterfactual explanation methods provide information on how feature values of individual observations must be changed to obtain a desired prediction.

counterfactual Counterfactual Explanation

Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetry

no code implementations6 Apr 2023 Jonas Gregor Wiese, Lisa Wimmer, Theodore Papamarkou, Bernd Bischl, Stephan Günnemann, David Rügamer

Bayesian inference in deep neural networks is challenging due to the high-dimensional, strongly multi-modal parameter posterior density landscape.

Bayesian Inference Uncertainty Quantification

Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis

2 code implementations20 Mar 2023 Tobias Weber, Michael Ingrisch, Bernd Bischl, David Rügamer

While recent advances in large-scale foundational models show promising results, their application to the medical domain has not yet been explored in detail.

Vocal Bursts Intensity Prediction

Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML

no code implementations15 Mar 2023 Hilde Weerts, Florian Pfisterer, Matthias Feurer, Katharina Eggensperger, Edward Bergman, Noor Awad, Joaquin Vanschoren, Mykola Pechenizkiy, Bernd Bischl, Frank Hutter

The field of automated machine learning (AutoML) introduces techniques that automate parts of the development of machine learning (ML) systems, accelerating the process and reducing barriers for novices.

AutoML Fairness

Representation Learning for Tablet and Paper Domain Adaptation in Favor of Online Handwriting Recognition

no code implementations16 Jan 2023 Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, Christopher Mutschler

The goal of domain adaptation (DA) is to mitigate this domain shift problem by searching for an optimal feature transformation to learn a domain-invariant representation.

Domain Adaptation Handwriting Recognition +1

Mind the Gap: Measuring Generalization Performance Across Multiple Objectives

1 code implementation8 Dec 2022 Matthias Feurer, Katharina Eggensperger, Edward Bergman, Florian Pfisterer, Bernd Bischl, Frank Hutter

Modern machine learning models are often constructed taking into account multiple objectives, e. g., minimizing inference time while also maximizing accuracy.

Hyperparameter Optimization

What cleaves? Is proteasomal cleavage prediction reaching a ceiling?

1 code implementation24 Oct 2022 Ingo Ziegler, Bolei Ma, Ercong Nie, Bernd Bischl, David Rügamer, Benjamin Schubert, Emilio Dorigatti

While direct identification of proteasomal cleavage \emph{in vitro} is cumbersome and low throughput, it is possible to implicitly infer cleavage events from the termini of MHC-presented epitopes, which can be detected in large amounts thanks to recent advances in high-throughput MHC ligandomics.

Benchmarking Denoising

Privacy-Preserving and Lossless Distributed Estimation of High-Dimensional Generalized Additive Mixed Models

1 code implementation14 Oct 2022 Daniel Schalk, Bernd Bischl, David Rügamer

In this paper, we propose an algorithm for a distributed, privacy-preserving, and lossless estimation of generalized additive mixed models (GAMM) using component-wise gradient boosting (CWB).

feature selection Privacy Preserving

Improved proteasomal cleavage prediction with positive-unlabeled learning

1 code implementation14 Sep 2022 Emilio Dorigatti, Bernd Bischl, Benjamin Schubert

Accurate in silico modeling of the antigen processing pathway is crucial to enable personalized epitope vaccine design for cancer.

Joint Debiased Representation and Image Clustering Learning with Self-Supervision

no code implementations14 Sep 2022 Shunjie-Fabian Zheng, JaeEun Nam, Emilio Dorigatti, Bernd Bischl, Shekoofeh Azizi, Mina Rezaei

However, existing methods for joint clustering and contrastive learning do not perform well on long-tailed data distributions, as majority classes overwhelm and distort the loss of minority classes, thus preventing meaningful representations to be learned.

Clustering Contrastive Learning +2

Robust and Efficient Imbalanced Positive-Unlabeled Learning with Self-supervision

1 code implementation6 Sep 2022 Emilio Dorigatti, Jonas Schweisthal, Bernd Bischl, Mina Rezaei

Learning from positive and unlabeled (PU) data is a setting where the learner only has access to positive and unlabeled samples while having no information on negative examples.

Knowledge Base Completion Medical Diagnosis +2

Tackling Neural Architecture Search With Quality Diversity Optimization

1 code implementation30 Jul 2022 Lennart Schneider, Florian Pfisterer, Paul Kent, Juergen Branke, Bernd Bischl, Janek Thomas

Although considerable progress has been made in the field of multi-objective NAS, we argue that there is some discrepancy between the actual optimization problem of practical interest and the optimization problem that multi-objective NAS tries to solve.

Neural Architecture Search

HPO X ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis

1 code implementation30 Jul 2022 Lennart Schneider, Lennart Schäpermeier, Raphael Patrick Prager, Bernd Bischl, Heike Trautmann, Pascal Kerschke

We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems.

Hyperparameter Optimization

AMLB: an AutoML Benchmark

2 code implementations25 Jul 2022 Pieter Gijsbers, Marcos L. P. Bueno, Stefan Coors, Erin LeDell, Sébastien Poirier, Janek Thomas, Bernd Bischl, Joaquin Vanschoren

Comparing different AutoML frameworks is notoriously challenging and often done incorrectly.

AutoML

FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation

1 code implementation31 May 2022 Mehmet Ozgur Turkoglu, Alexander Becker, Hüseyin Anil Gündüz, Mina Rezaei, Bernd Bischl, Rodrigo Caye Daudt, Stefano D'Aronco, Jan Dirk Wegner, Konrad Schindler

We show that the idea can be extended to uncertainty quantification: by modulating the network activations of a single deep network with FiLM, one obtains a model ensemble with high diversity, and consequently well-calibrated estimates of epistemic uncertainty, with low computational overhead in comparison.

Multi-Task Learning Probabilistic Deep Learning +1

What Is Fairness? On the Role of Protected Attributes and Fictitious Worlds

no code implementations19 May 2022 Ludwig Bothmann, Kristina Peters, Bernd Bischl

In this work, we try to bridge this gap by formalizing a consistent concept of fairness and by translating the philosophical considerations into a formal framework for the training and evaluation of ML models in ADM systems.

Decision Making Fairness

Efficient Automated Deep Learning for Time Series Forecasting

1 code implementation11 May 2022 Difan Deng, Florian Karl, Frank Hutter, Bernd Bischl, Marius Lindauer

In contrast to common NAS search spaces, we designed a novel neural architecture search space covering various state-of-the-art architectures, allowing for an efficient macro-search over different DL approaches.

Bayesian Optimization Neural Architecture Search +2

Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift

1 code implementation7 Apr 2022 Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, Christopher Mutschler

To mitigate this domain shift problem, domain adaptation (DA) techniques search for an optimal transformation that converts the (current) input data from a source domain to a target domain to learn a domain-invariant representation that reduces domain discrepancy.

Domain Adaptation Time Series +2

Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models

no code implementations4 Apr 2022 Ashkan Khakzar, Yawei Li, Yang Zhang, Mirac Sanisoglu, Seong Tae Kim, Mina Rezaei, Bernd Bischl, Nassir Navab

One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced.

Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition

no code implementations16 Feb 2022 Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, Christopher Mutschler

We perform extensive evaluations on synthetic image and time-series data, and on data for offline handwriting recognition (HWR) and on online HWR from sensor-enhanced pens for classifying written words.

Classification Handwriting Recognition +6

REPID: Regional Effect Plots with implicit Interaction Detection

1 code implementation15 Feb 2022 Julia Herbinger, Bernd Bischl, Giuseppe Casalicchio

Machine learning models can automatically learn complex relationships, such as non-linear and interaction effects.

BIG-bench Machine Learning Interpretable Machine Learning

Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens

no code implementations14 Feb 2022 Felix Ott, David Rügamer, Lucas Heublein, Tim Hamann, Jens Barth, Bernd Bischl, Christopher Mutschler

While there exist many offline HWR datasets, there is only little data available for the development of OnHWR methods on paper as it requires hardware-integrated pens.

Benchmarking Handwriting Recognition +1

DeepPAMM: Deep Piecewise Exponential Additive Mixed Models for Complex Hazard Structures in Survival Analysis

no code implementations12 Feb 2022 Philipp Kopper, Simon Wiegrebe, Bernd Bischl, Andreas Bender, David Rügamer

Survival analysis (SA) is an active field of research that is concerned with time-to-event outcomes and is prevalent in many domains, particularly biomedical applications.

Survival Analysis

Uncertainty-aware Pseudo-label Selection for Positive-Unlabeled Learning

no code implementations31 Jan 2022 Emilio Dorigatti, Jann Goschenhofer, Benjamin Schubert, Mina Rezaei, Bernd Bischl

In this work, we thus propose to tackle the issues of imbalanced datasets and model calibration in a PUL setting through an uncertainty-aware pseudo-labeling procedure (PUUPL): by boosting the signal from the minority class, pseudo-labeling expands the labeled dataset with new samples from the unlabeled set, while explicit uncertainty quantification prevents the emergence of harmful confirmation bias leading to increased predictive performance.

Pseudo Label Uncertainty Quantification

Marginal Effects for Non-Linear Prediction Functions

no code implementations21 Jan 2022 Christian A. Scholbeck, Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl, Christian Heumann

Hence, marginal effects are typically used as approximations for feature effects, either in the shape of derivatives of the prediction function or forward differences in prediction due to a change in a feature value.

Automated Benchmark-Driven Design and Explanation of Hyperparameter Optimizers

1 code implementation29 Nov 2021 Julia Moosbauer, Martin Binder, Lennart Schneider, Florian Pfisterer, Marc Becker, Michel Lang, Lars Kotthoff, Bernd Bischl

Automated hyperparameter optimization (HPO) has gained great popularity and is an important ingredient of most automated machine learning frameworks.

Bayesian Optimization Hyperparameter Optimization

Towards modelling hazard factors in unstructured data spaces using gradient-based latent interpolation

no code implementations21 Oct 2021 Tobias Weber, Michael Ingrisch, Bernd Bischl, David Rügamer

The application of deep learning in survival analysis (SA) allows utilizing unstructured and high-dimensional data types uncommon in traditional survival methods.

Decoder Survival Analysis

Survival-oriented embeddings for improving accessibility to complex data structures

no code implementations21 Oct 2021 Tobias Weber, Michael Ingrisch, Matthias Fabritius, Bernd Bischl, David Rügamer

We propose a hazard-regularized variational autoencoder that supports straightforward interpretation of deep neural architectures in the context of survival analysis, a field highly relevant in healthcare.

Decision Making Survival Analysis

Accelerated Componentwise Gradient Boosting using Efficient Data Representation and Momentum-based Optimization

no code implementations7 Oct 2021 Daniel Schalk, Bernd Bischl, David Rügamer

Componentwise boosting (CWB), also known as model-based boosting, is a variant of gradient boosting that builds on additive models as base learners to ensure interpretability.

Additive models

Self-GenomeNet: Self-supervised Learning with Reverse-Complement Context Prediction for Nucleotide-level Genomics Data

no code implementations29 Sep 2021 Hüseyin Anil Gündüz, Martin Binder, Xiao-Yin To, René Mreches, Philipp C. Münch, Alice C McHardy, Bernd Bischl, Mina Rezaei

We introduce Self-GenomeNet, a novel contrastive self-supervised learning method for nucleotide-level genomic data, which substantially improves the quality of the learned representations and performance compared to the current state-of-the-art deep learning frameworks.

Self-Supervised Learning

Deep Variational Clustering Framework for Self-labeling of Large-scale Medical Images

1 code implementation22 Sep 2021 Farzin Soleymani, Mohammad Eslami, Tobias Elze, Bernd Bischl, Mina Rezaei

We propose a Deep Variational Clustering (DVC) framework for unsupervised representation learning and clustering of large-scale medical images.

Clustering Decoder +1

Deep Bregman Divergence for Contrastive Learning of Visual Representations

no code implementations15 Sep 2021 Mina Rezaei, Farzin Soleymani, Bernd Bischl, Shekoofeh Azizi

In this paper, we propose deep Bregman divergences for contrastive learning of visual representation where we aim to enhance contrastive loss used in self-supervised learning by training additional networks based on functional Bregman divergence.

Contrastive Learning object-detection +2

Automatic Componentwise Boosting: An Interpretable AutoML System

no code implementations12 Sep 2021 Stefan Coors, Daniel Schalk, Bernd Bischl, David Rügamer

Despite its restriction to an interpretable model space, our system is competitive in terms of predictive performance on most data sets while being more user-friendly and transparent.

AutoML Feature Importance +2

Joint Debiased Representation Learning and Imbalanced Data Clustering

no code implementations11 Sep 2021 Mina Rezaei, Emilio Dorigatti, David Ruegamer, Bernd Bischl

We simultaneously train two deep learning models, a deep representation network that captures the data distribution, and a deep clustering network that learns embedded features and performs clustering.

Clustering Deep Clustering +2

YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization

1 code implementation8 Sep 2021 Florian Pfisterer, Lennart Schneider, Julia Moosbauer, Martin Binder, Bernd Bischl

When developing and analyzing new hyperparameter optimization methods, it is vital to empirically evaluate and compare them on well-curated benchmark suites.

Hyperparameter Optimization

Developing Open Source Educational Resources for Machine Learning and Data Science

no code implementations28 Jul 2021 Ludwig Bothmann, Sven Strickroth, Giuseppe Casalicchio, David Rügamer, Marius Lindauer, Fabian Scheipl, Bernd Bischl

It should be openly accessible to everyone, with as few barriers as possible; even more so for key technologies such as Machine Learning (ML) and Data Science (DS).

BIG-bench Machine Learning

Mutation is all you need

no code implementations ICML Workshop AutoML 2021 Lennart Schneider, Florian Pfisterer, Martin Binder, Bernd Bischl

Neural architecture search (NAS) promises to make deep learning accessible to non-experts by automating architecture engineering of deep neural networks.

Bayesian Optimization Neural Architecture Search

Decomposition of Global Feature Importance into Direct and Associative Components (DEDACT)

1 code implementation15 Jun 2021 Gunnar König, Timo Freiesleben, Bernd Bischl, Giuseppe Casalicchio, Moritz Grosse-Wentrup

Direct importance provides causal insight into the model's mechanism, yet it fails to expose the leakage of information from associated but not directly used variables.

Feature Importance

Meta-Learning for Symbolic Hyperparameter Defaults

1 code implementation10 Jun 2021 Pieter Gijsbers, Florian Pfisterer, Jan N. van Rijn, Bernd Bischl, Joaquin Vanschoren

Hyperparameter optimization in machine learning (ML) deals with the problem of empirically learning an optimal algorithm configuration from data, usually formulated as a black-box optimization problem.

Hyperparameter Optimization Meta-Learning

Grouped Feature Importance and Combined Features Effect Plot

1 code implementation23 Apr 2021 Quay Au, Julia Herbinger, Clemens Stachl, Bernd Bischl, Giuseppe Casalicchio

However, for researchers and practitioners, it is often equally important to quantify the importance or visualize the effect of feature groups.

BIG-bench Machine Learning Feature Importance +1

deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression

2 code implementations6 Apr 2021 David Rügamer, Chris Kolb, Cornelius Fritz, Florian Pfisterer, Philipp Kopper, Bernd Bischl, Ruolin Shen, Christina Bukas, Lisa Barros de Andrade e Sousa, Dominik Thalmeier, Philipp Baumann, Lucas Kook, Nadja Klein, Christian L. Müller

In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks.

regression

Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features

2 code implementations1 Apr 2021 Florian Pargent, Florian Pfisterer, Janek Thomas, Bernd Bischl

Since most machine learning (ML) algorithms are designed for numerical inputs, efficiently encoding categorical variables is a crucial aspect in data analysis.

BIG-bench Machine Learning

Deep Semi-Supervised Learning for Time Series Classification

1 code implementation6 Feb 2021 Jann Goschenhofer, Rasmus Hvingelby, David Rügamer, Janek Thomas, Moritz Wagner, Bernd Bischl

Based on these adaptations, we explore the potential of deep semi-supervised learning in the context of time series classification by evaluating our methods on large public time series classification problems with varying amounts of labelled samples.

Classification Data Augmentation +4

Semi-Structured Deep Piecewise Exponential Models

no code implementations11 Nov 2020 Philipp Kopper, Sebastian Pölsterl, Christian Wachinger, Bernd Bischl, Andreas Bender, David Rügamer

We propose a versatile framework for survival analysis that combines advanced concepts from statistics with deep learning.

Survival Analysis

Debiasing classifiers: is reality at variance with expectation?

no code implementations4 Nov 2020 Ashrya Agrawal, Florian Pfisterer, Bernd Bischl, Francois Buet-Golfouse, Srijan Sood, Jiahao Chen, Sameena Shah, Sebastian Vollmer

We present an empirical study of debiasing methods for classifiers, showing that debiasers often fail in practice to generalize out-of-sample, and can in fact make fairness worse rather than better.

Fairness

Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges

no code implementations19 Oct 2020 Christoph Molnar, Giuseppe Casalicchio, Bernd Bischl

To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

BIG-bench Machine Learning Causal Inference +1

Neural Mixture Distributional Regression

no code implementations14 Oct 2020 David Rügamer, Florian Pfisterer, Bernd Bischl

We present neural mixture distributional regression (NMDR), a holistic framework to estimate complex finite mixtures of distributional regressions defined by flexible additive predictors.

regression

Symplectic Gaussian Process Regression of Hamiltonian Flow Maps

no code implementations11 Sep 2020 Katharina Rath, Christopher G. Albert, Bernd Bischl, Udo von Toussaint

In the limit of small mapping times, the Hamiltonian function can be identified with a part of the generating function and thereby learned from observed time-series data of the system's evolution.

Numerical Integration regression +2

mlr3proba: An R Package for Machine Learning in Survival Analysis

no code implementations18 Aug 2020 Raphael Sonabend, Franz J. Király, Andreas Bender, Bernd Bischl, Michel Lang

As machine learning has become increasingly popular over the last few decades, so too has the number of machine learning interfaces for implementing these models.

Benchmarking BIG-bench Machine Learning +1

Relative Feature Importance

3 code implementations16 Jul 2020 Gunnar König, Christoph Molnar, Bernd Bischl, Moritz Grosse-Wentrup

Interpretable Machine Learning (IML) methods are used to gain insight into the relevance of a feature of interest for the performance of a model.

Feature Importance Interpretable Machine Learning

General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models

1 code implementation8 Jul 2020 Christoph Molnar, Gunnar König, Julia Herbinger, Timo Freiesleben, Susanne Dandl, Christian A. Scholbeck, Giuseppe Casalicchio, Moritz Grosse-Wentrup, Bernd Bischl

An increasing number of model-agnostic interpretation techniques for machine learning (ML) models such as partial dependence plots (PDP), permutation feature importance (PFI) and Shapley values provide insightful model interpretations, but can lead to wrong conclusions if applied incorrectly.

BIG-bench Machine Learning Feature Importance

A General Machine Learning Framework for Survival Analysis

1 code implementation27 Jun 2020 Andreas Bender, David Rügamer, Fabian Scheipl, Bernd Bischl

The modeling of time-to-event data, also known as survival analysis, requires specialized methods that can deal with censoring and truncation, time-varying features and effects, and that extend to settings with multiple competing events.

BIG-bench Machine Learning Data Augmentation +1

Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach

1 code implementation8 Jun 2020 Christoph Molnar, Gunnar König, Bernd Bischl, Giuseppe Casalicchio

In addition, we apply the conditional subgroups approach to partial dependence plots (PDP), a popular method for describing feature effects that can also suffer from extrapolation when features are dependent and interactions are present in the model.

Feature Importance

Multi-Objective Counterfactual Explanations

1 code implementation23 Apr 2020 Susanne Dandl, Christoph Molnar, Martin Binder, Bernd Bischl

We show the usefulness of MOC in concrete cases and compare our approach with state-of-the-art methods for counterfactual explanations.

counterfactual

Multi-Objective Hyperparameter Tuning and Feature Selection using Filter Ensembles

no code implementations30 Dec 2019 Martin Binder, Julia Moosbauer, Janek Thomas, Bernd Bischl

While model-based optimization needs fewer objective evaluations to achieve good performance, it incurs computational overhead compared to the NSGA-II, so the preferred choice depends on the cost of evaluating a model on given data.

feature selection Hyperparameter Optimization

Benchmarking time series classification -- Functional data vs machine learning approaches

1 code implementation18 Nov 2019 Florian Pfisterer, Laura Beggel, Xudong Sun, Fabian Scheipl, Bernd Bischl

In order to assess the methods and implementations, we run a benchmark on a wide variety of representative (time series) data sets, with in-depth analysis of empirical results, and strive to provide a reference ranking for which method(s) to use for non-expert practitioners.

Additive models Benchmarking +6

Towards Human Centered AutoML

no code implementations6 Nov 2019 Florian Pfisterer, Janek Thomas, Bernd Bischl

Building models from data is an integral part of the majority of data science workflows.

AutoML Position

Multi-Objective Automatic Machine Learning with AutoxgboostMC

no code implementations28 Aug 2019 Florian Pfisterer, Stefan Coors, Janek Thomas, Bernd Bischl

AutoML systems are currently rising in popularity, as they can build powerful models without human oversight.

AutoML BIG-bench Machine Learning +1

Tutorial and Survey on Probabilistic Graphical Model and Variational Inference in Deep Reinforcement Learning

no code implementations25 Aug 2019 Xudong Sun, Bernd Bischl

Aiming at a comprehensive and concise tutorial survey, recap of variational inference and reinforcement learning with Probabilistic Graphical Models are given with detailed derivations.

reinforcement-learning Reinforcement Learning (RL) +1

An Open Source AutoML Benchmark

no code implementations1 Jul 2019 Pieter Gijsbers, Erin LeDell, Janek Thomas, Sébastien Poirier, Bernd Bischl, Joaquin Vanschoren

In recent years, an active field of research has developed around automated machine learning (AutoML).

AutoML BIG-bench Machine Learning

Variational Resampling Based Assessment of Deep Neural Networks under Distribution Shift

1 code implementation7 Jun 2019 Xudong Sun, Alexej Gossmann, Yu Wang, Bernd Bischl

A novel variational inference based resampling framework is proposed to evaluate the robustness and generalization capability of deep learning models with respect to distribution shift.

Domain Generalization General Classification +3

Wearable-based Parkinson's Disease Severity Monitoring using Deep Learning

no code implementations24 Apr 2019 Jann Goschenhofer, Franz MJ Pfister, Kamer Ali Yuksel, Bernd Bischl, Urban Fietzek, Janek Thomas

To solve the problem of limited availability of high quality training data, we propose a transfer learning technique which helps to improve model performance substantially.

General Classification regression +4

ReinBo: Machine Learning pipeline search and configuration with Bayesian Optimization embedded Reinforcement Learning

1 code implementation10 Apr 2019 Xudong Sun, Jiali Lin, Bernd Bischl

Machine learning pipeline potentially consists of several stages of operations like data preprocessing, feature engineering and machine learning model training.

Bayesian Optimization BIG-bench Machine Learning +3

Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability

2 code implementations8 Apr 2019 Christoph Molnar, Giuseppe Casalicchio, Bernd Bischl

Post-hoc model-agnostic interpretation methods such as partial dependence plots can be employed to interpret complex machine learning models.

BIG-bench Machine Learning Interpretable Machine Learning

High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions

1 code implementation24 Feb 2019 Xudong Sun, Andrea Bommert, Florian Pfisterer, Jörg Rahnenführer, Michel Lang, Bernd Bischl

To carry out a clinical research under this scenario, an analyst could train a machine learning model only on local data site, but it is still possible to execute a statistical query at a certain cost in the form of sending a machine learning model to some of the remote data sites and get the performance measures as feedback, maybe due to prediction being usually much cheaper.

Bayesian Optimization BIG-bench Machine Learning +2

Robust Anomaly Detection in Images using Adversarial Autoencoders

no code implementations18 Jan 2019 Laura Beggel, Michael Pfeiffer, Bernd Bischl

Reliably detecting anomalies in a given set of images is a task of high practical relevance for visual quality inspection, surveillance, or medical image analysis.

Anomaly Detection

Automatic Gradient Boosting

3 code implementations10 Jul 2018 Janek Thomas, Stefan Coors, Bernd Bischl

Automatic machine learning performs predictive modeling with high performing machine learning tools without human interference.

BIG-bench Machine Learning Hyperparameter Optimization +1

Automatic Exploration of Machine Learning Experiments on OpenML

no code implementations28 Jun 2018 Daniel Kühn, Philipp Probst, Janek Thomas, Bernd Bischl

Understanding the influence of hyperparameters on the performance of a machine learning algorithm is an important scientific topic in itself and can help to improve automatic hyperparameter tuning procedures.

BIG-bench Machine Learning

Visualizing the Feature Importance for Black Box Models

1 code implementation18 Apr 2018 Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl

Based on local feature importance, we propose two visual tools: partial importance (PI) and individual conditional importance (ICI) plots which visualize how changes in a feature affect the model performance on average, as well as for individual observations.

Feature Importance

Tunability: Importance of Hyperparameters of Machine Learning Algorithms

2 code implementations26 Feb 2018 Philipp Probst, Bernd Bischl, Anne-Laure Boulesteix

Firstly, we formalize the problem of tuning from a statistical point of view, define data-based defaults and suggest general measures quantifying the tunability of hyperparameters of algorithms.

Benchmarking BIG-bench Machine Learning

mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions

4 code implementations9 Mar 2017 Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas, Michel Lang

We present mlrMBO, a flexible and comprehensive R toolbox for model-based optimization (MBO), also known as Bayesian optimization, which addresses the problem of expensive black-box optimization by approximating the given objective function through a surrogate regression model.

Bayesian Optimization regression +1

Probing for sparse and fast variable selection with model-based boosting

no code implementations15 Feb 2017 Janek Thomas, Tobias Hepp, Andreas Mayr, Bernd Bischl

We present a new variable selection method based on model-based gradient boosting and randomly permuted variables.

Variable Selection

Stability selection for component-wise gradient boosting in multiple dimensions

1 code implementation30 Nov 2016 Janek Thomas, Andreas Mayr, Bernd Bischl, Matthias Schmid, Adam Smith, Benjamin Hofner

We apply this new algorithm to a study to estimate abundance of common eider in Massachusetts, USA, featuring excess zeros, overdispersion, non-linearity and spatio-temporal structures.

Additive models

Fast model selection by limiting SVM training times

no code implementations10 Feb 2016 Aydin Demircioglu, Daniel Horn, Tobias Glasmachers, Bernd Bischl, Claus Weihs

Kernelized Support Vector Machines (SVMs) are among the best performing supervised learning methods.

Model Selection

ASlib: A Benchmark Library for Algorithm Selection

2 code implementations8 Jun 2015 Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Marius Lindauer, Yuri Malitsky, Alexandre Frechette, Holger Hoos, Frank Hutter, Kevin Leyton-Brown, Kevin Tierney, Joaquin Vanschoren

To address this problem, we introduce a standardized format for representing algorithm selection scenarios and a repository that contains a growing number of data sets from the literature.

OpenML: networked science in machine learning

1 code implementation29 Jul 2014 Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, Luis Torgo

Many sciences have made significant breakthroughs by adopting online tools that help organize, structure and mine information that is too detailed to be printed in journals.

BIG-bench Machine Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.