Search Results for author:
Found 8 papers, 4 papers with code
Refusal in Language Models Is Mediated by a Single Direction
In this work, we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size.
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
To study this problem, we organized a capture-the-flag competition at IEEE SaTML 2024, where the flag is a secret string in the LLM system prompt.
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
ARB: Advanced Reasoning Benchmark for Large Language Models
As a subset of ARB, we introduce a challenging set of math and physics problems which require advanced symbolic reasoning and domain knowledge.
Evaluating Superhuman Models with Consistency Checks
If machine learning models were to achieve superhuman abilities at various reasoning or decision-making tasks, how would we go about evaluating such models, given that humans would necessarily be poor proxies for ground truth?
Poisoning Web-Scale Training Datasets is Practical
Deep learning models are often trained on distributed, web-scale datasets crawled from the internet.
Red-Teaming the Stable Diffusion Safety Filter
We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content.
A law of adversarial risk, interpolation, and label noise
In supervised learning, it has been shown that label noise in the data can be interpolated without penalties on test accuracy.
