Search Results for author:
Found 10 papers, 3 papers with code
An Approach to Technical AGI Safety and Security
Second, system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned.
MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking
Future advanced AI systems may learn sophisticated strategies through reinforcement learning (RL) that humans cannot understand well enough to safely evaluate.
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
We primarily train SAEs on the Gemma 2 pre-trained models, but additionally release SAEs trained on instruction-tuned Gemma 2 9B for comparison.
Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders
To be useful for downstream tasks, SAEs need to decompose LM activations faithfully; yet to be interpretable the decomposition must be sparse -- two objectives that are in tension.
Improving Dictionary Learning with Gated Sparse Autoencoders
Recent work has found that sparse autoencoders (SAEs) are an effective technique for unsupervised discovery of interpretable features in language models' (LMs) activations, by finding sparse, linear reconstructions of LM activations.
Challenges with unsupervised LLM knowledge discovery
We show that existing unsupervised methods on large language model (LLM) activations do not discover knowledge -- instead they seem to discover whatever feature of the activations is most prominent.
Explaining grokking through circuit efficiency
One of the most surprising puzzles in neural network generalisation is grokking: a network with perfect training accuracy but poor generalisation will, upon further training, transition to perfect generalisation.
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
However, an AI system may pursue an undesired goal even when the specification is correct, in the case of goal misgeneralization.
Safe Deep RL in 3D Environments using Human Feedback
In this paper we answer this question in the affirmative, using ReQueST to train an agent to perform a 3D first-person object collection task using data entirely from human contractors.
Imitating Interactive Intelligence
These evaluations convincingly demonstrate that interactive training and auxiliary losses improve agent behaviour beyond what is achieved by supervised learning of actions alone.
