Search Results for author:
Found 9 papers, 3 papers with code
Investigating Sensitive Directions in GPT-2: An Improved Baseline and Comparative Analysis of SAEs
Sensitive directions experiments attempt to understand the computational features of Language Models (LMs) by measuring how much the next token prediction probabilities change by perturbing activations along specific directions.
Evolution of SAE Features Across Layers in LLMs
Sparse Autoencoders for transformer-based language models are typically defined independently per layer.
Characterizing stable regions in the residual stream of LLMs
We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region boundaries.
Evaluating Synthetic Activations composed of SAE Latents in GPT-2
In our study, we assess model sensitivity in order to compare real activations to synthetic activations composed of SAE latents.
You can remove GPT2's LayerNorm by fine-tuning
The LayerNorm (LN) layer in GPT-style transformer models has long been a hindrance to mechanistic interpretability.
The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
We present a novel interpretability method that aims to overcome this limitation by transforming the activations of the network into a new basis - the Local Interaction Basis (LIB).
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
We propose that if we can represent a neural network in a way that is invariant to reparameterizations that exploit the degeneracies, then this representation is likely to be more interpretable, and we provide some evidence that such a representation is likely to have sparser interactions.
How to use and interpret activation patching
Activation patching is a popular mechanistic interpretability technique, but has many subtleties regarding how it is applied and how one may interpret the results.
Towards Automated Circuit Discovery for Mechanistic Interpretability
For example, the ACDC algorithm rediscovered 5/5 of the component types in a circuit in GPT-2 Small that computes the Greater-Than operation.
