Search Results for author:
Found 9 papers, 1 papers with code
Adapting a Kidney Exchange Algorithm to Align with Human Values
In kidney exchanges, a central market maker allocates living kidney donors to patients in need of an organ.
Aligning with Heterogeneous Preferences for Kidney Exchange
In this paper, we propose, implement and evaluate a methodology for prioritizing patients based on such heterogeneous moral preferences.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Choice Set Misspecification in Reward Inference
A promising alternative to manually specifying reward functions is to enable robots to infer them from human feedback, like demonstrations or corrections.
The Expertise Problem: Learning from Specialized Feedback
RLHF algorithms that learn from multiple teachers therefore face an expertise problem: the reliability of a given piece of feedback depends both on the teacher that it comes from and how specialized that teacher is on relevant components of the task.
Active Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Active teacher selection for reinforcement learning from human feedback
The HUB framework and ATS algorithm demonstrate the importance of leveraging differences between teachers to learn accurate reward models, facilitating future research on active teacher selection for robust reward modeling.
Social Choice for AI Alignment: Dealing with Diverse Human Feedback
Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, so that, for example, they refuse to comply with requests for help with committing crimes or with producing racist text.
