Search Results for author:
Found 10 papers, 5 papers with code
Improving Dictionary Learning with Gated Sparse Autoencoders
Recent work has found that sparse autoencoders (SAEs) are an effective technique for unsupervised discovery of interpretable features in language models' (LMs) activations, by finding sparse, linear reconstructions of LM activations.
AtP*: An efficient and scalable method for localizing LLM behaviour to components
We investigate Attribution Patching (AtP), a fast gradient-based approximation to Activation Patching and find two classes of failure modes of AtP which lead to significant false negatives.
Explaining grokking through circuit efficiency
One of the most surprising puzzles in neural network generalisation is grokking: a network with perfect training accuracy but poor generalisation will, upon further training, transition to perfect generalisation.
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
\emph{Circuit analysis} is a promising technique for understanding the internal mechanisms of language models.
Tracr: Compiled Transformers as a Laboratory for Interpretability
Additionally, the known structure of Tracr-compiled models can serve as ground-truth for evaluating interpretability methods.
Learning to Play No-Press Diplomacy with Best Response Policy Iteration
It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms.
OpenSpiel: A Framework for Reinforcement Learning in Games
OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.
Learning Reciprocity in Complex Sequential Social Dilemmas
We analyse the resulting policies to show that the reciprocating agents are strongly influenced by their co-players' behavior.
Reinforcement and Imitation Learning for Diverse Visuomotor Skills
We propose a model-free deep reinforcement learning method that leverages a small amount of demonstration data to assist a reinforcement learning agent.
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
We propose zoneout, a novel method for regularizing RNNs.
