Search Results for author:
Found 29 papers, 9 papers with code
Mind the Graph When Balancing Data for Fairness or Robustness
Failures of fairness or robustness in machine learning predictive settings can be due to undesired dependencies between covariates, outcomes and auxiliary factors of variation.
The Impossibility of Fair LLMs
The need for fair AI is increasingly clear in the era of general-purpose systems such as ChatGPT, Gemini, and other large language models (LLMs).
Proxy Methods for Domain Adaptation
We study the problem of domain adaptation under distribution shift, where the shift is due to a change in the distribution of an unobserved, latent variable that confounds both the covariates and the labels.
CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?
Interestingly, data and architectural improvements seem to mitigate the negative impact of data balancing on performance; e. g. applying M4 to SigLIP-B/16 with data quality filters improves COCO image-to-text retrieval @5 from 86% (without data balancing) to 87% and ImageNet 0-shot classification from 77% to 77. 5%!
Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation
In this work, we study the correspondence between such decontextualized "trick tests" and evaluations that are more grounded in Realistic Use and Tangible {Effects (i. e. RUTEd evaluations).
Predictive Churn with the Set of Good Models
And we characterize expected churn over model updates via the Rashomon set, pairing our analysis with empirical results on real-world datasets -- showing how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
Theoretical guarantees on the best-of-n alignment policy
A commonly used analytical expression in the literature claims that the KL divergence between the best-of-$n$ policy and the base policy is equal to $\log (n) - (n-1)/n.$ We disprove the validity of this claim, and show that it is an upper bound on the actual KL divergence.
Choosing a Proxy Metric from Past Experiments
We introduce a new statistical framework to both define and construct an optimal proxy metric for use in a homogeneous population of randomized experiments.
When does Privileged Information Explain Away Label Noise?
Leveraging privileged information (PI), or features available during training but not at test time, has recently been shown to be an effective method for addressing label noise.
Adapting to Latent Subgroup Shifts via Concepts and Proxies
We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain, and unlabeled data from the target.
Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlations
However, we assume that the generative model for features $p(x|y, z)$ is invariant across domains.
Fairness and robustness in anti-causal prediction
Robustness to distribution shift and fairness have independently emerged as two important desiderata required of modern machine learning models.
Boosting the interpretability of clinical risk scores with intervention predictions
We propose a joint model of intervention policy and adverse event risk as a means to explicitly communicate the model's assumptions about future interventions.
Diagnosing failures of fairness transfer across distribution shift in real-world medical settings
Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings.
The MultiBERTs: BERT Reproductions for Robustness Analysis
Experiments with pre-trained models such as BERT are often based on a single checkpoint.
Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
Counterfactual Invariance to Spurious Correlations in Text Classification
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
Causally motivated Shortcut Removal Using Auxiliary Labels
Shortcut learning, in which models make use of easy-to-represent but unstable associations, is a major failure mode for robust machine learning.
Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap
We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data.
Revisiting Rashomon: A Comment on "The Two Cultures"
Here, I provide some reflections on Prof. Leo Breiman's "The Two Cultures" paper.
SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression
The key insight of SLOE is that the Sur and Cand\`es (2019) correction can be reparameterized in terms of the \emph{corrupted signal strength}, which is only a function of the estimated parameters $\widehat \beta$.
Copula-based Sensitivity Analysis for Multi-Treatment Causal Inference with Unobserved Confounding
Recent work has focused on the potential and pitfalls of causal identification in observational studies with multiple simultaneous treatments.
Causal Identification
Causal Inference
Methodology
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
On Robustness and Transferability of Convolutional Neural Networks
Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts.
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
Using this one line code change, we achieve state-of-the-art on recent covariate shift benchmarks and an mCE of 60. 28\% on the challenging ImageNet-C dataset; to our knowledge, this is the best result for any model that does not incorporate additional data augmentation or modification of the training pipeline.
A Biologically Plausible Benchmark for Contextual Bandit Algorithms in Precision Oncology Using in vitro Data
Here, we propose a benchmark dataset to evaluate contextual bandit algorithms based on real in vitro drug response of approximately 900 cancer cell lines.
Comment: Reflections on the Deconfounder
The aim of this comment (set to appear in a formal discussion in JASA) is to draw out some conclusions from an extended back-and-forth I have had with Wang and Blei regarding the deconfounder method proposed in "The Blessings of Multiple Causes" [arXiv:1805. 06826].
On Multi-Cause Causal Inference with Unobserved Confounding: Counterexamples, Impossibility, and Alternatives
Unobserved confounding is a central barrier to drawing causal inferences from observational data.
Reducing Reparameterization Gradient Variance
Optimization with noisy gradients has become ubiquitous in statistics and machine learning.
