Search Results for author:
Found 25 papers, 23 papers with code
Statistical Bias in Dataset Replication
Dataset replication is a useful tool for assessing whether models have overfit to a specific validation set or the exact circumstances under which it was generated.
DsDm: Model-Aware Dataset Selection with Datamodels
When selecting data for training large-scale models, standard practice is to filter for examples that match human notions of data quality.
FFCV: Accelerating Training by Removing Data Bottlenecks
For example, we are able to train an ImageNet ResNet-50 model to 75\% in only 20 mins on a single machine.
Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation
In this work, we introduce the notion of a dataset interface: a framework that, given an input dataset and a user-specified shift, returns instances from that input distribution that exhibit the desired shift.
When does Bias Transfer in Transfer Learning?
Using transfer learning to adapt a pre-trained "source model" to a downstream "target task" can dramatically increase performance with seemingly no downside.
Datamodels: Predicting Predictions from Training Data
We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data.
3DB: A Framework for Debugging Computer Vision Models
We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation.
Unadversarial Examples: Designing Objects for Robust Vision
We study a class of realistic computer vision settings wherein one can influence the design of the objects being recognized.
Do Adversarially Robust ImageNet Models Transfer Better?
Typically, better pre-trained models yield better transfer results, suggesting that initial accuracy is a key aspect of transfer learning performance.
Noise or Signal: The Role of Image Backgrounds in Object Recognition
We assess the tendency of state-of-the-art object recognition models to depend on signals from image backgrounds.
Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms: Proximal Policy Optimization (PPO) and Trust Region Policy Optimization (TRPO).
From ImageNet to Image Classification: Contextualizing Progress on Benchmarks
Building rich machine learning datasets in a scalable manner often necessitates a crowd-sourced data collection pipeline.
Identifying Statistical Bias in Dataset Replication
We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality.
Implementation Matters in Deep RL: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms, Proximal Policy Optimization and Trust Region Policy Optimization.
Image Synthesis with a Single (Robust) Classifier
We show that the basic classification framework alone can be used to tackle some of the most challenging tasks in image synthesis.
Ranked #60 on
Image Generation
on CIFAR-10
(Inception score metric)
Adversarial Robustness as a Prior for Learned Representations
In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks.
Adversarial Examples Are Not Bugs, They Are Features
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear.
A Closer Look at Deep Policy Gradients
We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development.
Evaluating and Understanding the Robustness of Adversarial Logit Pairing
We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples.
Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available.
Robustness May Be at Odds with Accuracy
We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization.
Black-box Adversarial Attacks with Limited Queries and Information
Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model.
Query-Efficient Black-box Adversarial Examples (superceded)
Second, we introduce a new algorithm to perform targeted adversarial attacks in the partial-information setting, where the attacker only has access to a limited number of target classes.
Exploring the Landscape of Spatial Robustness
The study of adversarial robustness has so far largely focused on perturbations bound in p-norms.
Synthesizing Robust Adversarial Examples
We demonstrate the existence of robust 3D adversarial objects, and we present the first algorithm for synthesizing examples that are adversarial over a chosen distribution of transformations.
