Search Results for author:
Found 8 papers, 2 papers with code
A StrongREJECT for Empty Jailbreaks
We show that our new grading scheme better accords with human judgment of response quality and overall jailbreak effectiveness, especially on the sort of low-quality responses that contribute the most to over-estimation of jailbreak performance on existing benchmarks.
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Our benchmark results show that many models are vulnerable to the attack strategies in the Tensor Trust dataset.
Active teacher selection for reinforcement learning from human feedback
The HUB framework and ATS algorithm demonstrate the importance of leveraging differences between teachers to learn accurate reward models, facilitating future research on active teacher selection for robust reward modeling.
Active Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system.
Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities
As automated decision making and decision assistance systems become common in everyday life, research on the prevention or mitigation of potential harms that arise from decisions made by these systems has proliferated.
Agent-aware State Estimation in Autonomous Vehicles
Autonomous systems often operate in environments where the behavior of multiple agents is coordinated by a shared global state.
Improving Competence for Reliable Autonomy
Given the complexity of real-world, unstructured domains, it is often impossible or impractical to design models that include every feature needed to handle all possible scenarios that an autonomous system may encounter.
Learning to Optimize Autonomy in Competence-Aware Systems
Interest in semi-autonomous systems (SAS) is growing rapidly as a paradigm to deploy autonomous systems in domains that require occasional reliance on humans.
