Search Results for author:
Found 9 papers, 4 papers with code
Training Neural Networks for Modularity aids Interpretability
An approach to improve network interpretability is via clusterability, i. e., splitting a model into disjoint clusters that can be studied independently.
Extending Activation Steering to Broad Skills and Multiple Behaviours
In this paper, we investigate the efficacy of activation steering for broad skills and multiple behaviours.
Dissecting Language Models: Machine Unlearning via Selective Pruning
This approach is a compute- and data-efficient method for identifying and removing neurons that enable specific behaviours.
Improving Activation Steering in Language Models with Mean-Centring
Recent work in activation steering has demonstrated the potential to better control the outputs of Large Language Models (LLMs), but it involves finding steering vectors.
Comparing Optimization Targets for Contrast-Consistent Search
We investigate the optimization target of Contrast-Consistent Search (CCS), which aims to recover the internal representations of truth of a large language model.
Any Deep ReLU Network is Shallow
We constructively prove that every deep ReLU network can be rewritten as a functionally identical three-layer network with weights valued in the extended reals.
Low-Entropy Latent Variables Hurt Out-of-Distribution Performance
We study the relationship between the entropy of intermediate representations and a model's robustness to distributional shift.
A theory of representation learning gives a deep generalisation of kernel methods
In particular, we show that Deep Gaussian processes (DGPs) in the Bayesian representation learning limit have exactly multivariate Gaussian posteriors, and the posterior covariances can be obtained by optimizing an interpretable objective combining a log-likelihood to improve performance with a series of KL-divergences which keep the posteriors close to the prior.
Learning to Communicate with Strangers via Channel Randomisation Methods
We introduce two methods for improving the performance of agents meeting for the first time to accomplish a communicative task.
