Search Results for author:
Found 17 papers, 8 papers with code
Measuring AI Ability to Complete Long Tasks
Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear.
Modular addition without black-boxes: Compressing explanations of MLPs that compute numerical integration
In this work, we present the first case study in rigorously compressing nonlinear feature-maps, which are the leading asymptotic bottleneck to compressing small transformer models.
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
We confirm that our experts make progress in the environments given 8 hours, with 82% of expert attempts achieving a non-zero score and 24% matching or exceeding our strong reference solutions.
Mathematical Models of Computation in Superposition
In this work, we present mathematical models of \emph{computation} in superposition, where superposition is actively helpful for efficiently accomplishing the task.
Compact Proofs of Model Performance via Mechanistic Interpretability
We propose using mechanistic interpretability -- techniques for reverse engineering model weights into human-interpretable algorithms -- to derive and compactly prove formal guarantees on model performance.
Evaluating Language-Model Agents on Realistic Autonomous Tasks
We find that these language model agents can only complete the easiest tasks from this list, although they make some progress on the more challenging tasks.
A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations
Universality is a key hypothesis in mechanistic interpretability -- that different models learn similar features and circuits when trained on similar tasks.
Progress measures for grokking via mechanistic interpretability
Based on this understanding, we define progress measures that allow us to study the dynamics of training and split training into three continuous phases: memorization, circuit formation, and cleanup.
Language models are better than humans at next-token prediction
Current language models are considered to have sub-human capabilities at natural language tasks like question-answering or writing code.
The Alignment Problem from a Deep Learning Perspective
In coming years or decades, artificial general intelligence (AGI) may surpass human capabilities at many critical tasks.
Adversarial Training for High-Stakes Reliability
We found that adversarial training increased robustness to the adversarial attacks that we trained on -- doubling the time for our contractors to find adversarial examples both with our tool (from 13 to 26 minutes) and without (from 20 to 44 minutes) -- without affecting in-distribution performance.
Human irrationality: both bad and good for reward inference
More importantly, we show that an irrational human, when correctly modelled, can communicate more information about the reward than a perfectly rational human can.
Optimal Cost Design for Model Predictive Control
We test our approach in an autonomous driving domain where we find costs different from the ground truth that implicitly compensate for replanning, short horizon, incorrect dynamics models, and local minima issues.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
The impacts of known and unknown demonstrator irrationality on reward inference
Surprisingly, we find that if we give the learner access to the correct model of the demonstrator's irrationality, these irrationalities can actually help reward inference.
Accounting for Human Learning when Inferring Human Preferences
Inverse reinforcement learning (IRL) is a common technique for inferring human preferences from data.
The Assistive Multi-Armed Bandit
Learning preferences implicit in the choices humans make is a well studied problem in both economics and computer science.
