Online Multivalid Learning: Means, Moments, and Prediction Intervals

no code implementations • 5 Jan 2021 • Varun Gupta, Christopher Jung, Georgy Noarov, Mallesh M. Pai, Aaron Roth

We present a general, efficient technique for providing contextual predictions that are "multivalid" in various senses, against an online sequence of adversarially chosen examples $(x, y)$.

Conformal Prediction Prediction Intervals

Moment Multicalibration for Uncertainty Estimation

no code implementations • 18 Aug 2020 • Christopher Jung, Changhwa Lee, Mallesh M. Pai, Aaron Roth, Rakesh Vohra

We show how to achieve the notion of "multicalibration" from H\'ebert-Johnson et al. [2018] not just for means, but also for variances and other higher moments.

Prediction Intervals valid

Fair Prediction with Endogenous Behavior

no code implementations • 18 Feb 2020 • Christopher Jung, Sampath Kannan, Changhwa Lee, Mallesh M. Pai, Aaron Roth, Rakesh Vohra

There is increasing regulatory interest in whether machine learning algorithms deployed in consequential domains (e. g. in criminal justice) treat different demographic groups "fairly."

BIG-bench Machine Learning Fairness

Competing Models

no code implementations • 8 Jul 2019 • Jose Luis Montiel Olea, Pietro Ortoleva, Mallesh M. Pai, Andrea Prat

Different agents need to make a prediction.

Model Selection Small Data Image Classification