Search Results for author:
Found 35 papers, 26 papers with code
Statistical Bias in Dataset Replication
Dataset replication is a useful tool for assessing whether models have overfit to a specific validation set or the exact circumstances under which it was generated.
Attribute-to-Delete: Machine Unlearning via Datamodel Matching
This perspective naturally suggests a reduction from the unlearning problem to that of data attribution, where the goal is to predict the effect of changing the training set on a model's outputs.
Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Machine learning models can fail on subgroups that are underrepresented during training.
Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content
Although this assumption is convenient, it fails to capture user strategization: that users may attempt to shape their future recommendations by adapting their behavior to the recommendation algorithm.
Decomposing and Editing Predictions by Modeling Model Computation
The goal of component modeling is to decompose an ML model's prediction in terms of its components -- simple functions (e. g., convolution filters, attention heads) that are the "building blocks" of model computation.
User Strategization and Trustworthy Algorithms
The developers of these algorithms commonly adopt the assumption that the data generating process is exogenous: that is, how a user reacts to a given prompt (e. g., a recommendation or hiring suggestion) depends on the prompt and not on the algorithm that generated it.
Rethinking Backdoor Attacks
In a backdoor attack, an adversary inserts maliciously constructed backdoor examples into a training set to make the resulting model vulnerable to manipulation.
FFCV: Accelerating Training by Removing Data Bottlenecks
For example, we are able to train an ImageNet ResNet-50 model to 75\% in only 20 mins on a single machine.
TRAK: Attributing Model Behavior at Scale
That is, computationally tractable methods can struggle with accurately attributing model predictions in non-convex settings (e. g., in the context of deep neural networks), while methods that are effective in such regimes require training thousands of models, which makes them impractical for large models or datasets.
Raising the Cost of Malicious AI-Powered Image Editing
We present an approach to mitigating the risks of malicious image editing posed by large diffusion models.
ModelDiff: A Framework for Comparing Learning Algorithms
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms.
When does Bias Transfer in Transfer Learning?
Using transfer learning to adapt a pre-trained "source model" to a downstream "target task" can dramatically increase performance with seemingly no downside.
What Makes A Good Fisherman? Linear Regression under Self-Selection Bias
In known-index self-selection, the identity of the observed model output is observable; in unknown-index self-selection, it is not.
Estimation of Standard Auction Models
We provide efficient estimation methods for first- and second-price auctions under independent (asymmetric) private values and partial observability.
Datamodels: Predicting Predictions from Training Data
We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data.
3DB: A Framework for Debugging Computer Vision Models
We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation.
Unadversarial Examples: Designing Objects for Robust Vision
We study a class of realistic computer vision settings wherein one can influence the design of the objects being recognized.
Do Adversarially Robust ImageNet Models Transfer Better?
Typically, better pre-trained models yield better transfer results, suggesting that initial accuracy is a key aspect of transfer learning performance.
Ranked #8 on
Object Recognition
on shape bias
Noise or Signal: The Role of Image Backgrounds in Object Recognition
We assess the tendency of state-of-the-art object recognition models to depend on signals from image backgrounds.
Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms: Proximal Policy Optimization (PPO) and Trust Region Policy Optimization (TRPO).
From ImageNet to Image Classification: Contextualizing Progress on Benchmarks
Building rich machine learning datasets in a scalable manner often necessitates a crowd-sourced data collection pipeline.
Identifying Statistical Bias in Dataset Replication
We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality.
Implementation Matters in Deep RL: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms, Proximal Policy Optimization and Trust Region Policy Optimization.
Image Synthesis with a Single (Robust) Classifier
We show that the basic classification framework alone can be used to tackle some of the most challenging tasks in image synthesis.
Adversarial Robustness as a Prior for Learned Representations
In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks.
Adversarial Examples Are Not Bugs, They Are Features
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear.
A Closer Look at Deep Policy Gradients
We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development.
Evaluating and Understanding the Robustness of Adversarial Logit Pairing
We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples.
Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available.
How Does Batch Normalization Help Optimization?
Batch Normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks (DNNs).
Black-box Adversarial Attacks with Limited Queries and Information
Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model.
The Robust Manifold Defense: Adversarial Training using Generative Models
Our formulation involves solving a min-max problem, where the min player sets the parameters of the classifier and the max player is running our attack, and is thus searching for adversarial examples in the {\em low-dimensional} input space of the spanner.
Query-Efficient Black-box Adversarial Examples (superceded)
Second, we introduce a new algorithm to perform targeted adversarial attacks in the partial-information setting, where the attacker only has access to a limited number of target classes.
Training GANs with Optimism
Moreover, we show that optimistic mirror decent addresses the limit cycling problem in training WGANs.
Synthesizing Robust Adversarial Examples
We demonstrate the existence of robust 3D adversarial objects, and we present the first algorithm for synthesizing examples that are adversarial over a chosen distribution of transformations.
