no code implementations • 14 Feb 2024 • Mira Jürgens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, Willem Waegeman
Trustworthy ML systems should not only return accurate predictions, but also a reliable representation of their uncertainty.
1 code implementation • 15 Sep 2023 • Nicolas Dewolf, Bernard De Baets, Willem Waegeman
Conformal prediction, and split conformal prediction as a specific implementation, offer a distribution-free approach to estimating prediction intervals with statistical guarantees.
1 code implementation • 10 Feb 2023 • Stijn Hawinkel, Willem Waegeman, Steven Maere
Out-of-sample prediction is the acid test of predictive models, yet an independent test dataset is often not available for assessment of the prediction error.
no code implementations • 30 Jan 2023 • Viktor Bengs, Eyke Hüllermeier, Willem Waegeman
In this paper, we generalise these findings and prove a more fundamental result: There seems to be no loss function that provides an incentive for a second-order learner to faithfully represent its epistemic uncertainty in the same manner as proper scoring rules do for standard (first-order) learners.
1 code implementation • 8 Nov 2022 • Dimitrios Iliadis, Marcel Wever, Bernard De Baets, Willem Waegeman
As a result of the ever increasing complexity of configuring and fine-tuning machine learning models, the field of automated machine learning (AutoML) has emerged over the past decade.
no code implementations • 20 May 2022 • Thomas Mortier, Viktor Bengs, Eyke Hüllermeier, Stijn Luca, Willem Waegeman
In this paper, we extend the notion of calibration, which is commonly used to evaluate the validity of the aleatoric uncertainty representation of a single probabilistic classifier, to assess the validity of an epistemic uncertainty representation obtained by sets of probabilistic classifiers.
no code implementations • 13 Mar 2022 • Thomas Mortier, Eyke Hüllermeier, Krzysztof Dembczyński, Willem Waegeman
Set-valued prediction is a well-known concept in multi-class classification.
no code implementations • 11 Mar 2022 • Viktor Bengs, Eyke Hüllermeier, Willem Waegeman
Uncertainty quantification has received increasing attention in machine learning in the recent past.
1 code implementation • 1 Jul 2021 • Nicolas Dewolf, Bernard De Baets, Willem Waegeman
Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods.
no code implementations • 19 Apr 2021 • Dimitrios Iliadis, Bernard De Baets, Willem Waegeman
In this work we present a generic deep learning methodology that can be used for a wide range of multi-target prediction problems.
1 code implementation • 21 Oct 2019 • Eyke Hüllermeier, Willem Waegeman
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology.
4 code implementations • 19 Jun 2019 • Thomas Mortier, Marek Wydmuch, Krzysztof Dembczyński, Eyke Hüllermeier, Willem Waegeman
In cases of uncertainty, a multi-class classifier preferably returns a set of candidate classes instead of predicting a single class label with little guarantee.
no code implementations • 7 Sep 2018 • Willem Waegeman, Krzysztof Dembczynski, Eyke Huellermeier
Multi-target prediction (MTP) is concerned with the simultaneous prediction of multiple target variables of diverse type.
no code implementations • 5 Mar 2018 • Michiel Stock, Tapio Pahikkala, Antti Airola, Bernard De Baets, Willem Waegeman
Problems of that kind are often referred to as pairwise learning, dyadic prediction or network inference problems.
no code implementations • 14 Jun 2016 • Michiel Stock, Krzysztof Dembczynski, Bernard De Baets, Willem Waegeman
Many complex multi-target prediction problems that concern large target spaces are characterised by a need for efficient prediction strategies that avoid the computation of predictions for all targets explicitly.
no code implementations • 14 Jun 2016 • Michiel Stock, Tapio Pahikkala, Antti Airola, Bernard De Baets, Willem Waegeman
In this work we analyze kernel-based methods for pairwise learning, with a particular focus on a recently-suggested two-step method.
no code implementations • 19 Jun 2015 • Tapio Pahikkala, Markus Viljanen, Antti Airola, Willem Waegeman
We consider the problem of learning regression functions from pairwise data when there exists prior knowledge that the relation to be learned is symmetric or anti-symmetric.
1 code implementation • 17 May 2014 • Tapio Pahikkala, Michiel Stock, Antti Airola, Tero Aittokallio, Bernard De Baets, Willem Waegeman
Dyadic prediction methods operate on pairs of objects (dyads), aiming to infer labels for out-of-sample dyads.
no code implementations • 17 May 2014 • Michiel Stock, Thomas Fober, Eyke Hüllermeier, Serghei Glinca, Gerhard Klebe, Tapio Pahikkala, Antti Airola, Bernard De Baets, Willem Waegeman
For a given query, the search operation results in a ranking of the enzymes in the database, from very similar to dissimilar enzymes, while information about the biological function of annotated database enzymes is ignored.
no code implementations • 17 Oct 2013 • Willem Waegeman, Krzysztof Dembczynski, Arkadiusz Jachnik, Weiwei Cheng, Eyke Hullermeier
The F-measure, which has originally been introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction.
no code implementations • NeurIPS 2012 • Weiwei Cheng, Eyke Hüllermeier, Willem Waegeman, Volkmar Welker
Several machine learning methods allow for abstaining from uncertain predictions.
no code implementations • 21 Sep 2012 • Tapio Pahikkala, Antti Airola, Michiel Stock, Bernard De Baets, Willem Waegeman
In domains like bioinformatics, information retrieval and social network analysis, one can find learning tasks where the goal consists of inferring a ranking of objects, conditioned on a particular target object.
no code implementations • NeurIPS 2011 • Krzysztof J. Dembczynski, Willem Waegeman, Weiwei Cheng, Eyke Hüllermeier
The F-measure, originally introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction.