Search Results for author:
Found 79 papers, 47 papers with code
Statistical Bias in Dataset Replication
Dataset replication is a useful tool for assessing whether models have overfit to a specific validation set or the exact circumstances under which it was generated.
Approaching Human-Level Forecasting with Language Models
In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters.
Feedback Loops With Language Models Drive In-Context Reward Hacking
Language models influence the external world: they query APIs that read and write to web pages, generate content that shapes human behavior, and run system commands as autonomous agents.
Describing Differences in Image Sets with Natural Language
To aid in this discovery process, we explore the task of automatically describing the differences between two $\textbf{sets}$ of images, which we term Set Difference Captioning.
How do Language Models Bind Entities in Context?
To correctly use in-context information, language models (LMs) must bind entities to their attributes.
Interpreting CLIP's Image Representation via Text-Based Decomposition
We decompose the image representation as a sum across individual image patches, model layers, and attention heads, and use CLIP's text representation to interpret the summands.
Overthinking the Truth: Understanding how Language Models Process False Demonstrations
The first phenomenon, overthinking, appears when we decode predictions from intermediate layers, given correct vs. incorrect few-shot demonstrations.
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
To answer these questions, we propose to evaluate $\textbf{counterfactual simulatability}$ of natural language explanations: whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals of the explained input.
Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations
This appears to suggest that the last-layer representations are completely determined by the labels, and do not depend on the intrinsic structure of input distribution.
Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition
As the scale of machine learning models increases, trends such as scaling laws anticipate consistent downstream improvements in predictive accuracy.
Mass-Producing Failures of Multimodal Systems with Language Models
Because CLIP is the backbone for most state-of-the-art multimodal systems, these inputs produce failures in Midjourney 5. 1, DALL-E, VideoFusion, and others.
Incentivizing High-Quality Content in Online Recommender Systems
For content recommender systems such as TikTok and YouTube, the platform's decision algorithm shapes the incentives of content producers, including how much effort the content producers invest in the quality of their content.
Eliciting Latent Predictions from Transformers with the Tuned Lens
We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer.
Automatically Auditing Large Language Models via Discrete Optimization
Auditing large language models for unexpected behaviors is critical to preempt catastrophic deployments, yet remains challenging.
Reward Learning as Doubly Nonparametric Bandits: Optimal Design and Scaling Laws
We specifically show that the well-studied problem of Gaussian process (GP) bandit optimization is a special case of our framework, and that our bounds either improve or are competitive with known regret guarantees for the Mat\'ern kernel.
Progress measures for grokking via mechanistic interpretability
Based on this understanding, we define progress measures that allow us to study the dynamics of training and split training into three continuous phases: memorization, circuit formation, and cleanup.
Discovering Latent Knowledge in Language Models Without Supervision
Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect.
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Research in mechanistic interpretability seeks to explain behaviors of machine learning models in terms of their internal components.
How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios
In experiments, we show how video models that are primarily trained to recognize actions and find contours of objects can be repurposed to understand human preferences and the emotional content of videos.
Forecasting Future World Events with Neural Networks
We test language models on our forecasting task and find that performance is far below a human expert baseline.
Auditing Visualizations: Transparency Methods Struggle to Detect Anomalous Behavior
Model visualizations provide information that outputs alone might miss.
Supply-Side Equilibria in Recommender Systems
Producers seek to create content that will be shown by the recommendation algorithm, which can impact both the diversity and quality of their content.
More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize
On the other hand, we find that the classical GCV estimator (Craven and Wahba, 1978) accurately predicts generalization risk even in such overparameterized settings.
Capturing Failures of Large Language Models via Human Cognitive Biases
Large language models generate complex, open-ended outputs: instead of outputting a class label they write summaries, generate dialogue, or produce working code.
Predicting Out-of-Distribution Error with the Projection Norm
Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels.
Describing Differences between Text Distributions with Natural Language
We then re-rank the descriptions by checking how often they hold on a larger set of samples with a learned verifier.
The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
Reward hacking -- where RL agents exploit gaps in misspecified reward functions -- has been widely observed, but not yet systematically studied.
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy.
The Effect of Model Size on Worst-Group Generalization
Overparameterization is shown to result in poor test accuracy on rare subgroups under a variety of settings where subgroup information is known.
Grounding Representation Similarity Through Statistical Testing
To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures.
What Would Jiminy Cricket Do? Towards Agents That Behave Morally
When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong.
Improving and Assessing Anomaly Detectors for Large-Scale Settings
We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.
Unsolved Problems in ML Safety
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings.
Learning Equilibria in Matching Markets from Bandit Feedback
Large-scale, two-sided matching platforms must find market outcomes that align with user preferences while simultaneously learning these preferences from data.
Grounding Representation Similarity with Statistical Testing
To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures.
Measuring Coding Challenge Competence With APPS
Recent models such as GPT-Neo can pass approximately 20% of the test cases of introductory problems, so we find that machine learning models are now beginning to learn how to code.
Ranked #8 on
Code Generation
on APPS
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level
We develop statistically rigorous methods to address this, and after accounting for pretraining and finetuning noise, we find that our BERT-Large is worse than BERT-Mini on at least 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%.
Agnostic learning with unknown utilities
This raises an interesting question whether learning is even possible in our setup, given that obtaining a generalizable estimate of utility $u^*$ might not be possible from finitely many samples.
Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
To investigate this gap, we decompose the test risk into its bias and variance components and study their behavior as a function of adversarial training perturbation radii ($\varepsilon$).
Approximating How Single Head Attention Learns
Our approximation explains why models sometimes attend to salient words, and inspires a toy example where a multi-head attention model can overcome the above hard training distribution by improving learning dynamics rather than expressiveness.
Limitations of Post-Hoc Feature Alignment for Robustness
Feature alignment is an approach to improving robustness to distribution shift that matches the distribution of feature activations between the training distribution and test distribution.
Measuring Mathematical Problem Solving With the MATH Dataset
To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
Ranked #95 on
Math Word Problem Solving
on MATH
A Rigorous Evaluation of Real-World Distribution Shifts
Motivated by this, we introduce a new data augmentation method which advances the state-of-the-art and outperforms models pretrained with 1000x more labeled data.
Towards Machine Ethics with Language Models
We show how to assess a language model’s knowledge of basic concepts of morality.
How Multipurpose Are Language Models?
By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
In this work, we propose a first-order dual SDP algorithm that (1) requires memory only linear in the total number of network activations, (2) only requires a fixed number of forward/backward passes through the network per iteration.
Measuring Massive Multitask Language Understanding
By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.
Ranked #60 on
Multi-task Language Understanding
on MMLU
Understanding Attention Training via Output Relevance
If we ablate attention by fixing it to uniform, the output relevance still correlates with the attention of a normally trained model; but if we instead ablate output relevance, attention cannot be learned.
Aligning AI With Shared Human Values
We show how to assess a language model's knowledge of basic concepts of morality.
Ranked #1 on
Average
on hendrycks2020ethics
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work.
Ranked #29 on
Domain Generalization
on ImageNet-R
Robust estimation via generalized quasi-gradients
We study the loss landscape of these robust estimation problems, and identify the existence of "generalized quasi-gradients".
Identifying Statistical Bias in Dataset Replication
We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality.
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks
We provide a simple explanation for this by measuring the bias and variance of neural networks: while the bias is monotonically decreasing as in the classical theory, the variance is unimodal or bell-shaped: it increases then decreases with the width of the network.
When does the Tukey median work?
We show that under TV corruptions, the breakdown point reduces to 1/4 for the same set of distributions.
Scaling Out-of-Distribution Detection for Real-World Settings
We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.
Generalized Resilience and Robust Statistics
This generalizes a property called resilience previously employed in the special case of mean estimation with outliers.
Testing Robustness Against Unforeseen Adversaries
To narrow in on this discrepancy between research and reality we introduce ImageNet-UA, a framework for evaluating model robustness against a range of unforeseen adversaries, including eighteen new non-L_p attacks.
Natural Adversarial Examples
We also curate an adversarial out-of-distribution detection dataset called ImageNet-O, which is the first out-of-distribution detection dataset created for ImageNet models.
Ranked #39 on
Domain Generalization
on ImageNet-A
Transfer of Adversarial Robustness Between Perturbation Types
We study the transfer of adversarial robustness of deep neural networks between different perturbation types.
FrAngel: Component-Based Synthesis with Control Structures
We present FrAngel, a new approach to component-based synthesis that can synthesize short Java functions with control structures when given a desired signature, a set of input-output examples, and a collection of libraries (without formal specifications).
Programming Languages
Semidefinite relaxations for certifying robustness to adversarial examples
One promise of ending the arms race is developing certified defenses, ones which are provably robust against all attackers in some family.
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
In this paper, we develop three attacks that can bypass a broad range of common data sanitization defenses, including anomaly detectors based on nearest neighbors, training loss, and singular-value decomposition.
Troubling Trends in Machine Learning Scholarship
Collectively, machine learning (ML) researchers are engaged in the creation and dissemination of knowledge about data-driven algorithms.
Sever: A Robust Meta-Algorithm for Stochastic Optimization
In high dimensions, most machine learning methods are brittle to even a small fraction of structured outliers.
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats.
Certified Defenses against Adversarial Examples
While neural networks have achieved high accuracy on standard image classification benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbations to test inputs.
Better Agnostic Clustering Via Relaxed Tensor Norms
As an immediate corollary, for any $\gamma > 0$, we obtain an efficient algorithm for learning the means of a mixture of $k$ arbitrary \Poincare distributions in $\mathbb{R}^d$ in time $d^{O(1/\gamma)}$ so long as the means have separation $\Omega(k^{\gamma})$.
Certified Defenses for Data Poisoning Attacks
Machine learning systems trained on user-provided data are susceptible to data poisoning attacks, whereby malicious users inject false training data with the aim of corrupting the learned model.
Does robustness imply tractability? A lower bound for planted clique in the semi-random model
This matches the conjectured computational threshold for the classical planted clique problem, and thus raises the intriguing possibility that, once we require robustness, there is no computational-statistical gap for planted clique.
Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers
We introduce a criterion, resilience, which allows properties of a dataset (such as its mean or best low rank approximation) to be robustly computed, even in the presence of a large fraction of arbitrary additional data.
Learning from Untrusted Data
For example, given a dataset of $n$ points for which an unknown subset of $\alpha n$ points are drawn from a distribution of interest, and no assumptions are made about the remaining $(1-\alpha)n$ points, is it possible to return a list of $\operatorname{poly}(1/\alpha)$ answers, one of which is correct?
Concrete Problems in AI Safety
Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society.
Unsupervised Risk Estimation Using Only Conditional Independence Structure
We show how to estimate a model's test error from unlabeled data, on distributions very different from the training distribution, while assuming only that certain conditional independencies are preserved between train and test.
Avoiding Imposters and Delinquents: Adversarial Crowdsourcing and Peer Prediction
We consider a crowdsourcing model in which $n$ workers are asked to rate the quality of $n$ items previously generated by other workers.
Learning with Relaxed Supervision
For weakly-supervised problems with deterministic constraints between the latent variables and observed output, learning necessitates performing inference over latent variables conditioned on the output, which can be intractable no matter how simple the model family is.
Learning Where to Sample in Structured Prediction
In structured prediction, most inference algorithms allocate a homogeneous amount of computation to all parts of the output, which can be wasteful when different parts vary widely in terms of difficulty.
Reified Context Models
A classic tension exists between exact inference in a simple model and approximate inference in a complex model.
Learning Fast-Mixing Models for Structured Prediction
Markov Chain Monte Carlo (MCMC) algorithms are often used for approximate inference inside learning, but their slow mixing can be difficult to diagnose and the approximations can seriously degrade learning.
The Statistics of Streaming Sparse Regression
We present a sparse analogue to stochastic gradient descent that is guaranteed to perform well under similar conditions to the lasso.
