Search Results for author:
Found 10 papers, 4 papers with code
SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
Agents are evaluated on their ability to complete the side task without appearing suspicious to an LLM monitor.
Unsupervised Elicitation of Language Models
Finally, we show that our method can improve the training of frontier LMs: we use our method to train an unsupervised reward model and use reinforcement learning to train a Claude 3. 5 Haiku-based assistant.
Best-of-N Jailbreaking
We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3. 5 Sonnet when sampling 10, 000 augmented prompts.
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem.
Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats
As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them.
Rapid Response: Mitigating LLM Jailbreaks with a Few Examples
We propose an alternative approach: instead of seeking perfect adversarial robustness, we develop rapid response techniques to look to block whole classes of jailbreaks after observing only a handful of attacks.
Looking Inward: Language Models Can Learn About Themselves by Introspection
We study introspection by finetuning LLMs to predict properties of their own behavior in hypothetical scenarios.
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
For example, the LLM red-teaming literature has produced a wide variety of 'jailbreaking' techniques to elicit harmful text from models that were fine-tuned to be harmless.
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
These results stand in stark contrast to existing evidence of universal and transferable text jailbreaks against language models and transferable adversarial attacks against image classifiers, suggesting that VLMs may be more robust to gradient-based transfer attacks.
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
