Search Results for author:
Found 17 papers, 7 papers with code
Forecasting Rare Language Model Behaviors
Standard language model evaluations can fail to capture risks that emerge only at deployment scale.
Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
Large language models (LLMs) are vulnerable to universal jailbreaks-prompting strategies that systematically bypass model safeguards and enable users to carry out harmful processes that require many model interactions, like manufacturing illegal substances at scale.
Best-of-N Jailbreaking
We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3. 5 Sonnet when sampling 10, 000 augmented prompts.
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem.
Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats
As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them.
Rapid Response: Mitigating LLM Jailbreaks with a Few Examples
We propose an alternative approach: instead of seeking perfect adversarial robustness, we develop rapid response techniques to look to block whole classes of jailbreaks after observing only a handful of attacks.
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks.
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
These results stand in stark contrast to existing evidence of universal and transferable text jailbreaks against language models and transferable adversarial attacks against image classifiers, suggesting that VLMs may be more robust to gradient-based transfer attacks.
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
Towards Understanding Sycophancy in Language Models
Overall, our results indicate that sycophancy is a general behavior of state-of-the-art AI assistants, likely driven in part by human preference judgments favoring sycophantic responses.
Understanding and Controlling a Maze-Solving Policy Network
To understand the goals and goal representations of AI systems, we carefully study a pretrained reinforcement learning policy that solves mazes by navigating to a range of target squares.
Incorporating Unlabelled Data into Bayesian Neural Networks
Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions.
Do Bayesian Neural Networks Need To Be Fully Stochastic?
We investigate the benefit of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard construction may be unnecessary.
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt
But most computation and time is wasted on redundant and noisy points that are already learnt or not learnable.
Prioritized training on points that are learnable, worth learning, and not yet learned (workshop version)
We introduce Goldilocks Selection, a technique for faster model training which selects a sequence of training points that are "just right".
How Robust are the Estimated Effects of Nonpharmaceutical Interventions against COVID-19?
To what extent are effectiveness estimates of nonpharmaceutical interventions (NPIs) against COVID-19 influenced by the assumptions our models make?
Differentially Private Federated Variational Inference
This setting is known as federated learning, in which privacy is a key concern.
