Classifier Calibration: A survey on how to assess and improve predicted class probabilities

no code implementations • 20 Dec 2021 • Telmo Silva Filho, Hao Song, Miquel Perello-Nieto, Raul Santos-Rodriguez, Meelis Kull, Peter Flach

This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration.

Classifier calibration Decision Making

Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration

1 code implementation • NeurIPS 2019 • Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, Peter Flach

Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence.

Binary Classification

Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration

3 code implementations • 28 Oct 2019 • Meelis Kull, Miquel Perello-Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, Peter Flach

Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence.

Binary Classification

$β^3$-IRT: A New Item Response Model and its Applications

1 code implementation • 10 Mar 2019 • Yu Chen, Telmo Silva Filho, Ricardo B. C. Prudêncio, Tom Diethe, Peter Flach

Item Response Theory (IRT) aims to assess latent abilities of respondents based on the correctness of their answers in aptitude test items with different difficulty levels.