Search Results for author:
Found 9 papers, 4 papers with code
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
AI Control: Improving Safety Despite Intentional Subversion
This protocol asks GPT-4 to write code, and then asks another instance of GPT-4 whether the code is backdoored, using various techniques to prevent the GPT-4 instances from colluding.
Generalized Wick Decompositions
We review the cumulant decomposition (a way of decomposing the expectation of a product of random variables (e. g. $\mathbb{E}[XYZ]$) into a sum of terms corresponding to partitions of these variables.)
Benchmarks for Detecting Measurement Tampering
When training powerful AI systems to perform complex tasks, it may be challenging to provide training signals which are robust to optimization.
Language models are better than humans at next-token prediction
Current language models are considered to have sub-human capabilities at natural language tasks like question-answering or writing code.
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Research in mechanistic interpretability seeks to explain behaviors of machine learning models in terms of their internal components.
Polysemanticity and Capacity in Neural Networks
We show that in a toy model the optimal capacity allocation tends to monosemantically represent the most important features, polysemantically represent less important features (in proportion to their impact on the loss), and entirely ignore the least important features.
Adversarial Training for High-Stakes Reliability
We found that adversarial training increased robustness to the adversarial attacks that we trained on -- doubling the time for our contractors to find adversarial examples both with our tool (from 13 to 26 minutes) and without (from 20 to 44 minutes) -- without affecting in-distribution performance.
Supervising strong learners by amplifying weak experts
Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior.
