Search Results for author:
Found 98 papers, 36 papers with code
Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules
Uncertainty representation and quantification are paramount in machine learning and constitute an important prerequisite for safety-critical applications.
Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration
We address this issue by proposing ShapleyBO, a framework for interpreting BO's proposals by game-theoretic Shapley values. They quantify each parameter's contribution to BO's acquisition function.
Conformalized Credal Set Predictors
Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution.
Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?
Trustworthy ML systems should not only return accurate predictions, but also a reliable representation of their uncertainty.
Information Leakage Detection through Approximate Bayes-optimal Prediction
To address these limitations, we establish a theoretical framework using statistical learning theory and information theory to accurately quantify and detect IL.
Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles
While shallow decision trees may be interpretable, larger ensemble models like gradient-boosted trees, which often set the state of the art in machine learning problems involving tabular data, still remain black box models.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
Second-Order Uncertainty Quantification: Variance-Based Measures
Uncertainty quantification is a critical aspect of machine learning models, providing important insights into the reliability of predictions and aiding the decision-making process in real-world applications.
A Survey of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a variant of reinforcement learning (RL) that learns from human feedback instead of relying on an engineered reward function.
Second-Order Uncertainty Quantification: A Distance-Based Approach
In the past couple of years, various approaches to representing and quantifying different types of predictive uncertainty in machine learning, notably in the setting of classification, have been proposed on the basis of second-order probability distributions, i. e., predictions in the form of distributions on probability distributions.
Continual Learning: Applications and the Road Forward
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past.
Identifying Copeland Winners in Dueling Bandits with Indifferences
We consider the task of identifying the Copeland winner(s) in a dueling bandits problem with ternary feedback.
Probabilistic Self-supervised Learning via Scoring Rules Minimization
In this paper, we propose a novel probabilistic self-supervised learning via Scoring Rule Minimization (ProSMIN), which leverages the power of probabilistic models to enhance representation quality and mitigate collapsing representations.
Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning
Ensembling a neural network is a widely recognized approach to enhance model performance, estimate uncertainty, and improve robustness in deep supervised learning.
Weighting by Tying: A New Approach to Weighted Rank Correlation
Measures of rank correlation are commonly used in statistics to capture the degree of concordance between two orderings of the same set of items.
A Novel Bayes' Theorem for Upper Probabilities
In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$, when the prior lies in a class of probability measures $\mathcal{P}$ and the likelihood is precise.
Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?
Adequate uncertainty representation and quantification have become imperative in various scientific disciplines, especially in machine learning and artificial intelligence.
iPDP: On Partial Dependence Plots in Dynamic Modeling Scenarios
Post-hoc explanation techniques such as the well-established partial dependence plot (PDP), which investigates feature dependencies, are used in explainable artificial intelligence (XAI) to understand black-box machine learning models.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
Conformal Prediction with Partially Labeled Data
While the predictions produced by conformal prediction are set-valued, the data used for training and calibration is supposed to be precise.
Koopman Kernel Regression
Many machine learning approaches for decision making, such as reinforcement learning, rely on simulators or predictive models to forecast the time-evolution of quantities of interest, e. g., the state of an agent or the reward of a policy.
Mitigating Label Noise through Data Ambiguation
Label noise poses an important challenge in machine learning, especially in deep learning, in which large models with high expressive power dominate the field.
Detecting Novelties with Empty Classes
We thereafter extend the DNN by $k$ empty classes and fine-tune it on the OoD data samples.
Optimizing Data Shapley Interaction Calculation from O(2^n) to O(t n^2) for KNN models
With the rapid growth of data availability and usage, quantifying the added value of each training data point has become a crucial process in the field of artificial intelligence.
iSAGE: An Incremental Version of SAGE for Online Explanation on Data Streams
Existing methods for explainable artificial intelligence (XAI), including popular feature importance measures such as SAGE, are mostly restricted to the batch learning scenario.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+2
Approximating the Shapley Value without Marginal Contributions
The Shapley value, which is arguably the most popular approach for assigning a meaningful contribution value to players in a cooperative game, has recently been used intensively in explainable artificial intelligence.
Iterative Deepening Hyperband
Hyperparameter optimization (HPO) is concerned with the automated search for the most appropriate hyperparameter configuration (HPC) of a parameterized machine learning algorithm.
On Second-Order Scoring Rules for Epistemic Uncertainty Quantification
In this paper, we generalise these findings and prove a more fundamental result: There seems to be no loss function that provides an incentive for a second-order learner to faithfully represent its epistemic uncertainty in the same manner as proper scoring rules do for standard (first-order) learners.
Conformal Prediction Intervals for Remaining Useful Lifetime Estimation
The main objective of Prognostics and Health Management is to estimate the Remaining Useful Lifetime (RUL), namely, the time that a system or a piece of equipment is still in working order before starting to function incorrectly.
AC-Band: A Combinatorial Bandit-Based Approach to Algorithm Configuration
We study the algorithm configuration (AC) problem, in which one seeks to find an optimal parameter configuration of a given target algorithm in an automated way.
Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning.
Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams
Explainable Artificial Intelligence (XAI) has mainly focused on static learning scenarios so far.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
Memorization-Dilation: Modeling Neural Collapse Under Label Noise
The notion of neural collapse refers to several emergent phenomena that have been empirically observed across various canonical classification problems.
Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark
On the one hand, the number of published xAI algorithms underwent a boom, and it became difficult for practitioners to select the right tool.
Conformal Credal Self-Supervised Learning
One such method, so-called credal self-supervised learning, maintains pseudo-supervision in the form of sets of (instead of single) probability distributions over labels, thereby allowing for a flexible yet uncertainty-aware labeling.
On the Calibration of Probabilistic Classifier Sets
In this paper, we extend the notion of calibration, which is commonly used to evaluate the validity of the aleatoric uncertainty representation of a single probabilistic classifier, to assess the validity of an epistemic uncertainty representation obtained by sets of probabilistic classifiers.
Set-valued prediction in hierarchical classification with constrained representation complexity
Set-valued prediction is a well-known concept in multi-class classification.
Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation
Uncertainty quantification has received increasing attention in machine learning in the recent past.
Finding Optimal Arms in Non-stochastic Combinatorial Bandits with Semi-bandit Feedback and Finite Budget
We consider the combinatorial bandits problem with semi-bandit feedback under finite sampling budget constraints, in which the learner can carry out its action only for a limited number of times specified by an overall budget.
Stochastic Contextual Dueling Bandits under Linear Stochastic Transitivity Models
In every round of the sequential decision problem, the learner makes a context-dependent selection of two choice alternatives (arms) to be compared with each other and receives feedback in the form of noisy preference information.
A Survey of Methods for Automated Algorithm Configuration
We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry.
Non-Stationary Dueling Bandits
We propose the $\mathrm{Beat\, the\, Winner\, Reset}$ algorithm and prove a bound on its expected binary weak regret in the stationary case, which tightens the bound of current state-of-art algorithms.
Prescriptive Machine Learning for Automated Decision Making: Challenges and Opportunities
Recent applications of machine learning (ML) reveal a noticeable shift from its use for predictive modeling in the sense of a data-driven construction of models mainly used for the purpose of prediction (of ground-truth facts) to its use for prescriptive modeling.
Identification of the Generalized Condorcet Winner in Multi-dueling Bandits
The reliable identification of the “best” arm while keeping the sample complexity as low as possible is a common task in the field of multi-armed bandits.
Towards Green Automated Machine Learning: Status Quo and Future Directions
Therefore, we first elaborate on how to quantify the environmental footprint of an AutoML tool.
Machine Learning for Online Algorithm Selection under Censored Feedback
In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms.
Automated Machine Learning, Bounded Rationality, and Rational Metareasoning
The notion of bounded rationality originated from the insight that perfectly rational behavior cannot be realized by agents with limited cognitive or computational resources.
iART: A Search Engine for Art-Historical Images to Support Research in the Humanities
In this paper, we introduce iART: an open Web platform for art-historical research that facilitates the process of comparative vision.
Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years.
Algorithm Selection on a Meta Level
The problem of selecting an algorithm that appears most suitable for a specific instance of an algorithmic problem class, such as the Boolean satisfiability problem, is called instance-specific algorithm selection.
Gradient-based Label Binning in Multi-label Classification
Based on the derivatives computed during training, we dynamically group the labels into a predefined number of bins to impose an upper bound on the dimensionality of the linear system.
Credal Self-Supervised Learning
In our approach, we therefore allow the learner to label instances in the form of credal sets, that is, sets of (candidate) probability distributions.
Annotation Uncertainty in the Context of Grammatical Change
This paper elaborates on the notion of uncertainty in the context of annotation in large text corpora, specifically focusing on (but not limited to) historical languages.
Ranking Structured Objects with Graph Neural Networks
Graph neural networks (GNNs) have been successfully applied in many structured data domains, with applications ranging from molecular property prediction to the analysis of social networks.
Ranked #1 on
Graph Ranking
on ZINC
Efficient time stepping for numerical integration using reinforcement learning
While the classical schemes apply very generally and are highly efficient on regular systems, they can behave sub-optimal when an inefficient step rejection mechanism is triggered by structurally complex systems such as chaotic systems.
Learning Structured Declarative Rule Sets -- A Challenge for Deep Discrete Learning
Arguably the key reason for the success of deep neural networks is their ability to autonomously form non-linear combinations of the input features, which can be used in subsequent layers of the network.
Towards Meta-Algorithm Selection
Instance-specific algorithm selection (AS) deals with the automatic selection of an algorithm from a fixed set of candidates most suitable for a specific instance of an algorithmic problem class, where "suitability" often refers to an algorithm's runtime.
A Flexible Class of Dependence-aware Multi-Label Loss Functions
For evaluating such predictions, the set of predicted labels needs to be compared to the ground-truth label set associated with that instance, and various loss functions have been proposed for this purpose.
Multi-Armed Bandits with Censored Consumption of Resources
We consider a resource-aware variant of the classical multi-armed bandit problem: In each round, the learner selects an arm and determines a resource limit.
Monocular Depth Estimation via Listwise Ranking using the Plackett-Luce Model
In many real-world applications, the relative depth of objects in an image is crucial for scene understanding.
Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis
Deep Q-Learning is an important reinforcement learning algorithm, which involves training a deep neural network, called Deep Q-Network (DQN), to approximate the well-known Q-function.
Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction
Syntactic annotation of corpora in the form of part-of-speech (POS) tags is a key requirement for both linguistic research and subsequent automated natural language processing (NLP) tasks.
Conformal Rule-Based Multi-label Classification
We advocate the use of conformal prediction (CP) to enhance rule-based multi-label classification (MLC).
Learning Choice Functions via Pareto-Embeddings
We consider the problem of learning to choose from a given set of objects, where each object is represented by a feature vector.
Machine Learning with the Sugeno Integral: The Case of Binary Classification
More specifically, we propose a method for binary classification, in which the Sugeno integral is used as an aggregation function that combines several local evaluations of an instance, pertaining to different features or measurements, into a single global evaluation.
Run2Survive: A Decision-theoretic Approach to Algorithm Selection based on Survival Analysis
In an extensive experimental study with the standard benchmark ASlib, our approach is shown to be highly competitive and in many cases even superior to state-of-the-art AS approaches.
A Novel Higher-order Weisfeiler-Lehman Graph Convolution
Current GNN architectures use a vertex neighborhood aggregation scheme, which limits their discriminative power to that of the 1-dimensional Weisfeiler-Lehman (WL) graph isomorphism test.
Ranked #6 on
Graph Classification
on REDDIT-B
Learning Gradient Boosted Multi-label Classification Rules
In multi-label classification, where the evaluation of predictions is less straightforward than in single-label classification, various meaningful, though different, loss functions have been proposed.
On Aggregation in Ensembles of Multilabel Classifiers
While a variety of ensemble methods for multilabel classification have been proposed in the literature, the question of how to aggregate the predictions of the individual members of the ensemble has received little attention so far.
Towards Analogy-Based Explanations in Machine Learning
Principles of analogical reasoning have recently been applied in the context of machine learning, for example to develop new methods for classification and preference learning.
Towards a Scalable and Flexible Simulation and Testing Environment Toolbox for Intelligent Microgrid Control
Micro- and smart grids (MSG) play an important role both for integrating renewable energy sources in conventional electricity grids and for providing power supply in remote areas.
Systems and Control Systems and Control
Online Preselection with Context Information under the Plackett-Luce Model
We consider an extension of the contextual multi-armed bandit problem, in which, instead of selecting a single alternative (arm), a learner is supposed to make a preselection in the form of a subset of alternatives.
Extreme Algorithm Selection With Dyadic Feature Representation
Algorithm selection (AS) deals with selecting an algorithm from a fixed set of candidate algorithms most suitable for a specific instance of an algorithmic problem, e. g., choosing solvers for SAT problems.
Aleatoric and Epistemic Uncertainty with Random Forests
In particular, the idea of distinguishing between two important types of uncertainty, often refereed to as aleatoric and epistemic, has recently been studied in the setting of supervised learning.
TSK-Streams: Learning TSK Fuzzy Systems on Data Streams
The problem of adaptive learning from evolving and possibly non-stationary data streams has attracted a lot of interest in machine learning in the recent past, and also stimulated research in related fields, such as computational intelligence and fuzzy systems.
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology.
Epistemic Uncertainty Sampling
In this paper, we advocate a distinction between two different types of uncertainty, referred to as epistemic and aleatoric, in the context of active learning.
Preselection Bandits
To formalize this goal, we introduce a reasonable notion of regret and derive lower bounds on the expected regret.
Efficient Set-Valued Prediction in Multi-Class Classification
In cases of uncertainty, a multi-class classifier preferably returns a set of candidate classes instead of predicting a single class label with little guarantee.
Rectifying Classifier Chains for Multi-Label Classification
Classifier chains have recently been proposed as an appealing method for tackling the multi-label classification task.
Reliable Multi-label Classification: Prediction with Partial Abstention
In contrast to conventional (single-label) classification, the setting of multilabel classification (MLC) allows an instance to belong to several classes simultaneously.
Learning Context-Dependent Choice Functions
Choice functions accept a set of alternatives as input and produce a preferred subset of these alternatives as output.
Analogy-Based Preference Learning with Kernels
Building on a specific formalization of analogical relationships of the form "A relates to B as C relates to D", we establish a connection between two important subfields of artificial intelligence, namely analogical reasoning and kernel-based machine learning.
Learning Interpretable Rules for Multi-label Classification
Multi-label classification (MLC) is a supervised learning problem in which, contrary to standard multiclass classification, an instance can be associated with several class labels simultaneously.
Automated Multi-Label Classification based on ML-Plan
Automated machine learning (AutoML) has received increasing attention in the recent past.
Preference-based Online Learning with Dueling Bandits: A Survey
The aim of this paper is to provide a survey of the state of the art in this field, referred to as preference-based multi-armed bandits or dueling bandits.
ML-Plan: Automated machine learning via hierarchical planning
Automated machine learning (AutoML) seeks to automatically select, compose, and parametrize machine learning algorithms, so as to achieve optimal performance on a given task (dataset).
Ranking Distributions based on Noisy Sorting
We also introduce a generalization of the model, in which the constraints on pairwise preferences are relaxed, and for which maximum likelihood estimation can be carried out based on a variation of the generalized iterative scaling algorithm.
Mining Rank Data
The problem of frequent pattern mining has been studied quite extensively for various types of data, including sets, sequences, and graphs.
Deep Architectures for Learning Context-dependent Ranking Functions
Object ranking is an important problem in the realm of preference learning.
Statistical Inference for Incomplete Ranking Data: The Case of Rank-Dependent Coarsening
We consider the problem of statistical inference for ranking data, specifically rank aggregation, under the assumption that samples are incomplete in the sense of not comprising all choice alternatives.
From knowledge-based to data-driven modeling of fuzzy rule-based systems: A critical reflection
This paper briefly elaborates on a development in (applied) fuzzy logic that has taken place in the last couple of decades, namely, the complementation or even replacement of the traditional knowledge-based approach to fuzzy rule-based systems design by a data-driven one.
Learning to Rank based on Analogical Reasoning
In this paper, we propose a new approach to object ranking based on principles of analogical reasoning.
Predicting Rankings of Software Verification Competitions
Software verification competitions, such as the annual SV-COMP, evaluate software verification tools with respect to their effectivity and efficiency.
Online F-Measure Optimization
In this paper, we study the problem of F-measure maximization in the setting of online learning.
Online Rank Elicitation for Plackett-Luce: A Dueling Bandits Approach
We study the problem of online rank elicitation, assuming that rankings of a set of alternatives obey the Plackett-Luce distribution.
Identification of functionally related enzymes by learning-to-rank methods
For a given query, the search operation results in a ranking of the enzymes in the database, from very similar to dissimilar enzymes, while information about the biological function of annotated database enzymes is ignored.
Learning from Imprecise and Fuzzy Observations: Data Disambiguation through Generalized Loss Minimization
Methods for analyzing or learning from "fuzzy data" have attracted increasing attention in recent years.
Label Ranking with Partial Abstention based on Thresholded Probabilistic Models
Several machine learning methods allow for abstaining from uncertain predictions.
An Exact Algorithm for F-Measure Maximization
The F-measure, originally introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction.
