Search Results for author:
Found 24 papers, 11 papers with code
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability.
The Bias of Harmful Label Associations in Vision-Language Models
Despite the remarkable performance of foundation vision-language models, the shared representation space for text and vision can also encode harmful label associations detrimental to fairness.
WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Models
We propose WorldSense, a benchmark designed to assess the extent to which LLMs are consistently able to sustain tacit world models, by testing how they draw simple inferences from descriptions of simple arrangements of entities.
Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations
Self-supervised representation learning often uses data augmentations to induce some invariance to "style" attributes of the data.
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks
Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more.
Discovering environments with XRM
Successful out-of-distribution generalization requires environment annotations.
VPA: Fully Test-Time Visual Prompt Adaptation
Inspired by the success of textual prompting, several studies have investigated the efficacy of visual prompt tuning.
PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning
Synthetic image datasets offer unmatched advantages for designing and evaluating deep neural networks: they make it possible to (i) render as many data samples as needed, (ii) precisely control each scene and yield granular ground truth labels (and captions), (iii) precisely control distribution shifts between training and testing to isolate variables of interest for sound experimentation.
Does Progress On Object Recognition Benchmarks Improve Real-World Generalization?
Second, we study model generalization across geographies by measuring the disparities in performance across regions, a more fine-grained measure of real world generalization.
A Cookbook of Self-Supervised Learning
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning.
Pinpointing Why Object Recognition Performance Degrades Across Income Levels and Geographies
As an example, we show that mitigating a model's vulnerability to texture can improve performance on the lower income level.
A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others
Key to advancing the reliability of vision systems is understanding whether existing methods can overcome multiple shortcuts or struggle in a Whac-A-Mole game, i. e., where mitigating one shortcut amplifies reliance on others.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations
Equipped with ImageNet-X, we investigate 2, 200 current recognition models and study the types of mistakes as a function of model's (1) architecture, e. g. transformer vs. convolutional, (2) learning paradigm, e. g. supervised vs. self-supervised, and (3) training procedures, e. g., data augmentation.
Robust Self-Supervised Learning with Lie Groups
Our approach applies the formalism of Lie groups to capture continuous transformations to improve models' robustness to distributional shifts.
The Robustness Limits of SoTA Vision Models to Natural Variation
We study not only how robust recent state-of-the-art models are, but also the extent to which models can generalize variation in factors when they're present during training.
Disentanglement of Correlated Factors via Hausdorff Factorized Support
We show that the use of HFS consistently facilitates disentanglement and recovery of ground-truth factors across a variety of correlation settings and benchmarks, even under severe training correlations and correlation shifts, with in parts over $+60\%$ in relative improvement over existing disentanglement methods.
CrypTen: Secure Multi-Party Computation Meets Machine Learning
To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks.
Grounding inductive biases in natural images:invariance stems from variations in data
While prior work has focused on synthetic data, we attempt here to characterize the factors of variation in a real dataset, ImageNet, and study the invariance of both standard residual networks and the recently proposed vision transformer with respect to changes in these factors.
Grounding inductive biases in natural images: invariance stems from variations in data
While prior work has focused on synthetic data, we attempt here to characterize the factors of variation in a real dataset, ImageNet, and study the invariance of both standard residual networks and the recently proposed vision transformer with respect to changes in these factors.
Addressing the Topological Defects of Disentanglement via Distributed Operators
A core challenge in Machine Learning is to learn to disentangle natural factors of variation in data (e. g. object shape vs. pose).
Addressing the Topological Defects of Disentanglement
A core challenge in Machine Learning is to disentangle natural factors of variation in data (e. g. object shape vs pose).
Global Explanations of Neural Networks: Mapping the Landscape of Predictions
A barrier to the wider adoption of neural networks is their lack of interpretability.
Mixed Membership Recurrent Neural Networks
Models for sequential data such as the recurrent neural network (RNN) often implicitly model a sequence as having a fixed time interval between observations and do not account for group-level effects when multiple sequences are observed.
Towards Explainable Deep Learning for Credit Lending: A Case Study
Deep learning adoption in the financial services industry has been limited due to a lack of model interpretability.
