Search Results for author:
Found 24 papers, 18 papers with code
Prompt-to-Leaderboard
To address this, we propose Prompt-to-Leaderboard (P2L), a method that produces leaderboards specific to a prompt.
Gradient Equilibrium in Online Learning: Theory and Applications
We present a new perspective on online learning that we refer to as gradient equilibrium: a sequence of iterates achieves gradient equilibrium if the average of gradients of losses along the sequence converges to zero.
Theoretical Foundations of Conformal Prediction
This book is about conformal prediction and related inferential techniques that build on permutation tests and exchangeability.
How to Evaluate Reward Models for RLHF
To address this, we build a predictive model of downstream LLM performance by evaluating the reward model on proxy tasks.
Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Decision-making pipelines are generally characterized by tradeoffs among various risk functions.
AutoEval Done Right: Using Synthetic Data for Model Evaluation
The evaluation of machine learning models using human-labeled validation data can be expensive and time-consuming.
Wavefront Randomization Improves Deconvolution
The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image.
Online conformal prediction with decaying step sizes
We introduce a method for online conformal prediction with decaying step sizes.
PPI++: Efficient Prediction-Powered Inference
We present PPI++: a computationally lightweight methodology for estimation and inference based on a small labeled dataset and a typically much larger dataset of machine-learning predictions.
Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions.
Conformal PID Control for Time Series Prediction
We study the problem of uncertainty quantification for time series prediction, with the goal of providing easy-to-use algorithms with formal guarantees.
Class-Conditional Conformal Prediction with Many Classes
Standard conformal prediction methods provide a marginal coverage guarantee, which means that for a random test point, the conformal prediction set contains the true label with a user-specified probability.
Prediction-Powered Inference
Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system.
Label Noise Robustness of Conformal Prediction
We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise.
Conformal Risk Control
We extend conformal prediction to control the expected value of any monotone loss function.
Semantic uncertainty intervals for disentangled latent spaces
Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street.
Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets
Our work applies these new uncertainty quantification methods -- specifically conformal prediction -- to a deep-learning model for grading the severity of spinal stenosis in lumbar spine MRI.
Recommendation Systems with Distribution-Free Reliability Guarantees
Building from a pre-trained ranking model, we show how to return a set of items that is rigorously guaranteed to contain mostly good items.
Ring deconvolution microscopy: exploiting symmetry for efficient spatially varying aberration correction
The most ubiquitous form of computational aberration correction for microscopy is deconvolution.
Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design
This is challenging because of a characteristic type of distribution shift between the training and test data in the design setting -- one in which the training and test data are statistically dependent, as the latter is chosen based on the former.
Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees.
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models.
Private Prediction Sets
Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine.
Event Based, Near Eye Gaze Tracking Beyond 10,000Hz
The cameras in modern gaze-tracking systems suffer from fundamental bandwidth and power limitations, constraining data acquisition speed to 300 Hz realistically.
