Search Results for author:
Found 31 papers, 21 papers with code
Theoretical Foundations of Conformal Prediction
This book is about conformal prediction and related inferential techniques that build on permutation tests and exchangeability.
Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Decision-making pipelines are generally characterized by tradeoffs among various risk functions.
Online conformal prediction with decaying step sizes
We introduce a method for online conformal prediction with decaying step sizes.
Delegating Data Collection in Decentralized Machine Learning
Motivated by the emergence of decentralized machine learning (ML) ecosystems, we study the delegation of data collection.
Incentive-Theoretic Bayesian Inference for Collaborative Science
We show how the principal can conduct statistical inference that leverages the information that is revealed by an agent's strategic behavior -- their choice to run a trial or not.
Class-Conditional Conformal Prediction with Many Classes
Standard conformal prediction methods provide a marginal coverage guarantee, which means that for a random test point, the conformal prediction set contains the true label with a user-specified probability.
Operationalizing Counterfactual Metrics: Incentives, Ranking, and Information Asymmetry
From the social sciences to machine learning, it has been well documented that metrics to be optimized are not always aligned with social welfare.
Prediction-Powered Inference
Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system.
The Sample Complexity of Online Contract Design
This result shows that exponential-in-$m$ samples are sufficient and necessary to learn a near-optimal contract, resolving an open problem on the hardness of online contract design.
Label Noise Robustness of Conformal Prediction
We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise.
Conformal Risk Control
We extend conformal prediction to control the expected value of any monotone loss function.
Semantic uncertainty intervals for disentangled latent spaces
Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street.
Recommendation Systems with Distribution-Free Reliability Guarantees
Building from a pre-trained ranking model, we show how to return a set of items that is rigorously guaranteed to contain mostly good items.
Robust Calibration with Multi-domain Temperature Scaling
Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains.
Achieving Risk Control in Online Learning Settings
To provide rigorous uncertainty quantification for online learning models, we develop a framework for constructing uncertainty sets that provably control risk -- such as coverage of confidence intervals, false negative rate, or F1 score -- in the online setting.
Principal-Agent Hypothesis Testing
The efficacy of the drug is not known to the regulator, so the pharmaceutical company must run a costly trial to prove efficacy to the regulator.
Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging
Image-to-image regression is an important learning task, used frequently in biological imaging.
Conformal prediction for the design problem
This is challenging because of a characteristic type of distribution shift between the training and test data in the design setting -- one in which the training and test data are statistically dependent, as the latter is chosen based on the former.
Optimal Data Selection: An Online Distributed View
The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets.
Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees.
Calibrated Multiple-Output Quantile Regression with Representation Learning
We develop a method to generate predictive regions that cover a multivariate response variable with a user-specified probability.
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models.
Test-time Collective Prediction
An increasingly common setting in machine learning involves multiple parties, each with their own data, who want to jointly make predictions on future test points.
Improving Conditional Coverage via Orthogonal Quantile Regression
To remedy this, we modify the loss function to promote independence between the size of the intervals and the indicator of a miscoverage event.
Testing for Outliers with Conformal p-values
We then introduce a new method to compute p-values that are both valid conditionally on the training data and independent of each other for different test points; this paves the way to stronger type-I error guarantees.
Cross-validation: what does it estimate and how well does it do it?
Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood.
Private Prediction Sets
Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine.
Distribution-Free, Risk-Controlling Prediction Sets
While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making.
Uncertainty Sets for Image Classifiers using Conformal Prediction
Convolutional image classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, hindering their deployment in consequential settings.
Metropolized Knockoff Sampling
Model-X knockoffs is a wrapper that transforms essentially any feature importance measure into a variable selection algorithm, which discovers true effects while rigorously controlling the expected fraction of false positives.
Methodology
