Search Results for author:
Found 9 papers, 3 papers with code
AssistanceZero: Scalably Solving Assistance Games
We present the first scalable approach to solving assistance games and apply it to a new, challenging Minecraft-based assistance game with over $10^{400}$ possible goals.
A StrongREJECT for Empty Jailbreaks
To create a benchmark, researchers must choose a dataset of forbidden prompts to which a victim model will respond, along with an evaluation method that scores the harmfulness of the victim model's responses.
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Our benchmark results show that many models are vulnerable to the attack strategies in the Tensor Trust dataset.
Active teacher selection for reinforcement learning from human feedback
The HUB framework and ATS algorithm demonstrate the importance of leveraging differences between teachers to learn accurate reward models, facilitating future research on active teacher selection for robust reward modeling.
Active Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system.
Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities
As automated decision making and decision assistance systems become common in everyday life, research on the prevention or mitigation of potential harms that arise from decisions made by these systems has proliferated.
Agent-aware State Estimation in Autonomous Vehicles
Autonomous systems often operate in environments where the behavior of multiple agents is coordinated by a shared global state.
Improving Competence for Reliable Autonomy
Given the complexity of real-world, unstructured domains, it is often impossible or impractical to design models that include every feature needed to handle all possible scenarios that an autonomous system may encounter.
Learning to Optimize Autonomy in Competence-Aware Systems
Interest in semi-autonomous systems (SAS) is growing rapidly as a paradigm to deploy autonomous systems in domains that require occasional reliance on humans.
