Search Results for author:
Found 13 papers, 8 papers with code
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
To study this problem, we organized a capture-the-flag competition at IEEE SaTML 2024, where the flag is a secret string in the LLM system prompt.
Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs
Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities.
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
Attributions toward Artificial Agents in a modified Moral Turing Test
Advances in artificial intelligence (AI) raise important questions about whether people view moral evaluations by AI systems similarly to human-generated moral evaluations.
Universal Jailbreak Backdoors from Poisoned Human Feedback
Reinforcement Learning from Human Feedback (RLHF) is used to align large language models to produce helpful and harmless responses.
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
Despite efforts to align large language models to produce harmless responses, they are still vulnerable to jailbreak prompts that elicit unrestricted behaviour.
Personas as a Way to Model Truthfulness in Language Models
This allows the model to separate truth from falsehoods and controls the truthfulness of its generation.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
PassGPT: Password Modeling and (Guided) Generation with Large Language Models
Large language models (LLMs) successfully model natural language from vast amounts of text without the need for explicit supervision.
Red-Teaming the Stable Diffusion Safety Filter
We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content.
Exploring Adversarial Attacks and Defenses in Vision Transformers trained with DINO
This work conducts the first analysis on the robustness against adversarial attacks on self-supervised Vision Transformers trained using DINO.
"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks
Adversarial attacks are a major challenge faced by current machine learning research.
Uneven Coverage of Natural Disasters in Wikipedia: the Case of Flood
We also note how coverage of floods in countries with the lowest income, as well as countries in South America, is substantially lower than the coverage of floods in middle-income countries.
