Search Results for author:
Found 35 papers, 18 papers with code
Beyond Linear Steering: Unified Multi-Attribute Control for Language Models
Controlling multiple behavioral attributes in large language models (LLMs) at inference time is a challenging problem due to interference between attributes and the limitations of linear steering methods, which assume additive behavior in activation space and require per-attribute tuning.
SafetyNet: Detecting Harmful Outputs in LLMs by Modeling and Monitoring Deceptive Behaviors
We propose a real-time framework to predict harmful AI outputs before they occur by using an unsupervised approach that treats normal behavior as the baseline and harmful outputs as outliers.
Scaling sparse feature circuit finding for in-context learning
In this work, we demonstrate their effectiveness by using SAEs to deepen our understanding of the mechanism behind in-context learning (ICL).
Do Sparse Autoencoders Generalize? A Case Study of Answerability
We extensively evaluate SAE feature generalization across diverse answerability datasets for Gemma 2 SAEs.
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
This work represents a crucial step toward establishing global standards for AI risk and reliability evaluation while acknowledging the need for continued development in areas such as multiturn interactions, multimodal understanding, coverage of additional languages, and emerging hazard categories.
Rethinking AI Cultural Alignment
As general-purpose artificial intelligence (AI) systems become increasingly integrated with diverse human communities, cultural alignment has emerged as a crucial element in their deployment.
Open Problems in Machine Unlearning for AI Safety
As AI systems become more capable, widely deployed, and increasingly autonomous in critical areas such as cybersecurity, biological research, and healthcare, ensuring their safety and alignment with human values is paramount.
Best-of-N Jailbreaking
We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3. 5 Sonnet when sampling 10, 000 augmented prompts.
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem.
Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders
By training on synthetic data with known features of the input, we show that \textsc{MFR} can help SAEs learn those features, as we can directly compare the features learned by the SAE with the input features for the synthetic data.
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks.
Towards Interpreting Visual Information Processing in Vision-Language Models
Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions.
Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders
Providing evidence for this hypothesis would enable researchers to exploit universal properties, facilitating the generalization of mechanistic interpretability techniques across models.
Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
We construct a curriculum of increasingly sophisticated gameable environments and find that training on early-curriculum environments leads to more specification gaming on remaining environments.
Risks and Opportunities of Open-Source Generative AI
Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education.
Visualizing Neural Network Imagination
Our goal is to visualize what environment states the networks are representing.
Near to Mid-term Risks and Opportunities of Open-Source Generative AI
In the next few years, applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education.
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
Understanding the inner workings of large language models (LLMs) is crucial for advancing their theoretical foundations and real-world applications.
Increasing Trust in Language Models through the Reuse of Verified Circuits
To exhibit the reusability of verified modules, we insert the trained integer addition model into a larger untrained model and train the combined model to perform both addition and subtraction.
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
Large Language Models Relearn Removed Concepts
This demonstrates that models exhibit polysemantic capacities and can blend old and new concepts in individual neurons.
Measuring Value Alignment
As artificial intelligence (AI) systems become increasingly integrated into various domains, ensuring that they align with human values becomes critical.
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
We extend this research by analyzing and comparing circuits for similar sequence continuation tasks, which include increasing sequences of Arabic numerals, number words, and months.
Understanding Addition in Transformers
Understanding the inner workings of machine learning models like Transformers is vital for their safe and ethical use.
Interpreting Learned Feedback Patterns in Large Language Models
Our probes are trained on a condensed, sparse and interpretable representation of LLM activations, making it easier to correlate features of the input with our probe's predictions.
AI Systems of Concern
Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning.
DeepDecipher: Accessing and Investigating Neuron Activation in Large Language Models
By granting access to state-of-the-art interpretability methods, DeepDecipher makes LLMs more transparent, trustworthy, and safe.
Neuron to Graph: Interpreting Language Model Neurons at Scale
Conventional methods require examination of examples with strong neuron activation and manual identification of patterns to decipher the concepts a neuron responds to.
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
We use this improved benchmark to evaluate recent model editing techniques and find that they suffer from low specificity.
The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python
Large Language Models (LLMs) have successfully been applied to code generation tasks, raising the question of how well these models understand programming.
System III: Learning with Domain Knowledge for Safety Constraints
We evaluate the satisfaction of these constraints via p-norms in state vector space.
N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models
Understanding the function of individual neurons within language models is essential for mechanistic interpretability research.
Fairness in AI and Its Long-Term Implications on Society
Successful deployment of artificial intelligence (AI) in various settings has led to numerous positive outcomes for individuals and society.
PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
