Search Results for author:
Found 5 papers, 3 papers with code
Circuit Breaking: Removing Model Behaviors with Targeted Ablation
Language models often exhibit behaviors that improve performance on a pre-training objective but harm performance on downstream tasks.
Benchmarks for Detecting Measurement Tampering
When training powerful AI systems to perform complex tasks, it may be challenging to provide training signals which are robust to optimization.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Discovering Variable Binding Circuitry with Desiderata
Recent work has shown that computation in language models may be human-understandable, with successful efforts to localize and intervene on both single-unit features and input-output circuits.
Robust Feature-Level Adversaries are Interpretability Tools
We demonstrate that they can be used to produce targeted, universal, disguised, physically-realizable, and black-box attacks at the ImageNet scale.
