Search Results for author:
Found 6 papers, 4 papers with code
Successor Heads: Recurring, Interpretable Attention Heads In The Wild
In this work we present successor heads: attention heads that increment tokens with a natural ordering, such as numbers, months, and days.
Attribution Patching Outperforms Automated Circuit Discovery
Automated interpretability research has recently attracted attention as a potential research direction that could scale explanations of neural network behavior to large models.
Copy Suppression: Comprehensively Understanding an Attention Head
We show that self-repair is implemented by several mechanisms, one of which is copy suppression, which explains 39% of the behavior in a narrow task.
Towards Automated Circuit Discovery for Mechanistic Interpretability
For example, the ACDC algorithm rediscovered 5/5 of the component types in a circuit in GPT-2 Small that computes the Greater-Than operation.
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Research in mechanistic interpretability seeks to explain behaviors of machine learning models in terms of their internal components.
StyleGAN-induced data-driven regularization for inverse problems
Our proposed approach, which we refer to as learned Bayesian reconstruction with generative models (L-BRGM), entails joint optimization over the style-code and the input latent code, and enhances the expressive power of a pre-trained StyleGAN2 generator by allowing the style-codes to be different for different generator layers.
