Search Results for author:
Found 9 papers, 1 papers with code
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text.
Active teacher selection for reinforcement learning from human feedback
The HUB framework and ATS algorithm demonstrate the importance of leveraging differences between teachers to learn accurate reward models, facilitating future research on active teacher selection for robust reward modeling.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Active Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system.
The Expertise Problem: Learning from Specialized Feedback
RLHF algorithms that learn from multiple teachers therefore face an expertise problem: the reliability of a given piece of feedback depends both on the teacher that it comes from and how specialized that teacher is on relevant components of the task.
Choice Set Misspecification in Reward Inference
A promising alternative to manually specifying reward functions is to enable robots to infer them from human feedback, like demonstrations or corrections.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Aligning with Heterogeneous Preferences for Kidney Exchange
In this paper, we propose, implement and evaluate a methodology for prioritizing patients based on such heterogeneous moral preferences.
Adapting a Kidney Exchange Algorithm to Align with Human Values
In kidney exchanges, a central market maker allocates living kidney donors to patients in need of an organ.
