Search Results for author:
Found 23 papers, 19 papers with code
Statistical Bias in Dataset Replication
Dataset replication is a useful tool for assessing whether models have overfit to a specific validation set or the exact circumstances under which it was generated.
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
To make progress towards understanding in-context learning, we consider the well-defined problem of training a model to in-context learn a function class (e. g., linear functions): that is, given data derived from some functions in the class, can we train a model to in-context learn "most" functions from this class?
Editing a classifier by rewriting its prediction rules
We present a methodology for modifying the behavior of a classifier by directly rewriting its prediction rules.
Combining Diverse Feature Priors
To improve model generalization, model designers often restrict the features that their models use, either implicitly or explicitly.
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance.
BREEDS: Benchmarks for Subpopulation Shift
We develop a methodology for assessing the robustness of models to subpopulation shift---specifically, their ability to generalize to novel data subpopulations that were not observed during training.
Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms: Proximal Policy Optimization (PPO) and Trust Region Policy Optimization (TRPO).
From ImageNet to Image Classification: Contextualizing Progress on Benchmarks
Building rich machine learning datasets in a scalable manner often necessitates a crowd-sourced data collection pipeline.
Identifying Statistical Bias in Dataset Replication
We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality.
Implementation Matters in Deep RL: A Case Study on PPO and TRPO
We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms, Proximal Policy Optimization and Trust Region Policy Optimization.
Label-Consistent Backdoor Attacks
While such attacks are very effective, they crucially rely on the adversary injecting arbitrary inputs that are---often blatantly---mislabeled.
Image Synthesis with a Single (Robust) Classifier
We show that the basic classification framework alone can be used to tackle some of the most challenging tasks in image synthesis.
Ranked #56 on
Image Generation
on CIFAR-10
(Inception score metric)
Adversarial Robustness as a Prior for Learned Representations
In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks.
Adversarial Examples Are Not Bugs, They Are Features
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear.
Clean-Label Backdoor Attacks
Deep neural networks have been recently demonstrated to be vulnerable to backdoor attacks.
On Evaluating Adversarial Robustness
Correctly evaluating defenses against adversarial examples has proven to be extremely difficult.
A Closer Look at Deep Policy Gradients
We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development.
Robustness May Be at Odds with Accuracy
We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization.
How Does Batch Normalization Help Optimization?
Batch Normalization (BatchNorm) is a widely adopted technique that enables faster and more stable training of deep neural networks (DNNs).
Adversarially Robust Generalization Requires More Data
We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
Exploring the Landscape of Spatial Robustness
The study of adversarial robustness has so far largely focused on perturbations bound in p-norms.
Towards Deep Learning Models Resistant to Adversarial Attacks
Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal.
