Search Results for author:
Found 22 papers, 11 papers with code
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
The SaTML '24 CNN Interpretability Competition: New Innovations for Concept-Level Interpretability
Interpretability techniques are valuable for helping humans understand and oversee AI systems.
Defending Against Unforeseen Failure Modes with Latent Adversarial Training
Despite extensive diagnostics and debugging by developers, AI systems sometimes exhibit harmful unintended behaviors.
Eight Methods to Evaluate Robust Unlearning in LLMs
Machine unlearning can be useful for removing harmful capabilities and memorized text from large language models (LLMs), but there are not yet standardized methods for rigorously evaluating it.
Rethinking Machine Unlearning for Large Language Models
We explore machine unlearning (MU) in the domain of large language models (LLMs), referred to as LLM unlearning.
Black-Box Access is Insufficient for Rigorous AI Audits
The effectiveness of an audit, however, depends on the degree of system access granted to auditors.
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?
This has led some researchers to conclude that LMs "lie" or otherwise encode non-cooperative communicative intents.
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
Despite efforts to align large language models to produce harmless responses, they are still vulnerable to jailbreak prompts that elicit unrestricted behaviour.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Measuring the Success of Diffusion Models at Imitating Human Artists
When Stable Diffusion is prompted to imitate an artist from this set, we find that the artist can be identified from the imitation with an average accuracy of 81. 0%.
Explore, Establish, Exploit: Red Teaming Language Models from Scratch
Using a pre-existing classifier does not allow for red-teaming to be tailored to the target model.
Diagnostics for Deep Neural Networks with Automated Copy/Paste Attacks
Some previous works have proposed using human-interpretable adversarial attacks including copy/paste attacks in which one natural image pasted into another causes an unexpected misclassification.
Red Teaming with Mind Reading: White-Box Adversarial Policies Against RL Agents
In this work, we study white-box adversarial policies and show that having access to a target agent's internal state can be useful for identifying its vulnerabilities.
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
The last decade of machine learning has seen drastic increases in scale and capabilities.
Quantifying Local Specialization in Deep Neural Networks
These results suggest that graph-based partitioning can reveal local specialization and that statistical methods can be used to automatedly screen for sets of neurons that can be understood abstractly.
Robust Feature-Level Adversaries are Interpretability Tools
We demonstrate that they can be used to produce targeted, universal, disguised, physically-realizable, and black-box attacks at the ImageNet scale.
Detecting Modularity in Deep Neural Networks
These results suggest that graph-based partitioning can reveal modularity and help us understand how deep neural networks function.
Clusterability in Neural Networks
We also exhibit novel methods to promote clusterability in neural network training, and find that in multi-layer perceptrons they lead to more clusterable networks with little reduction in accuracy.
Importance and Coherence: Methods for Evaluating Modularity in Neural Networks
We apply these methods on partitionings generated by a spectral clustering algorithm which uses a graph representation of the network's neurons and weights.
Achilles Heels for AGI/ASI via Decision Theoretic Adversaries
As progress in AI continues to advance, it is important to know how advanced systems will make choices and in what ways they may fail.
Probing Neural Dialog Models for Conversational Understanding
The predominant approach to open-domain dialog generation relies on end-to-end training of neural models on chat datasets.
Frivolous Units: Wider Networks Are Not Really That Wide
We identify two distinct types of "frivolous" units that proliferate when the network's width is increased: prunable units which can be dropped out of the network without significant change to the output and redundant units whose activities can be expressed as a linear combination of others.
