Dataset replication is a useful tool for assessing whether models have overfit to a specific validation set or the exact circumstances under which it was generated.
We present a methodology for modifying the behavior of a classifier by directly rewriting its prediction rules.
As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance.
We develop a methodology for assessing the robustness of models to subpopulation shift---specifically, their ability to generalize to novel data subpopulations that were not observed during training.
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms: Proximal Policy Optimization (PPO) and Trust Region Policy Optimization (TRPO).
Building rich machine learning datasets in a scalable manner often necessitates a crowd-sourced data collection pipeline.
We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality.
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms, Proximal Policy Optimization and Trust Region Policy Optimization.
We show that the basic classification framework alone can be used to tackle some of the most challenging tasks in image synthesis.
Ranked #53 on Image Generation on CIFAR-10 (Inception score metric)
In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks.
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear.
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult.
We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development.
We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization.
We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
The study of adversarial robustness has so far largely focused on perturbations bound in p-norms.
Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal.
Ranked #2 on Robust classification on CIFAR-10