Search Results for author:
Found 11 papers, 2 papers with code
Beyond Preferences in AI Alignment
We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values.
AI Alignment with Changing and Influenceable Reward Functions
Existing AI alignment approaches assume that preferences are static, which is unrealistic: our preferences change, and may even be influenced by our interactions with AI systems themselves.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Who Needs to Know? Minimal Knowledge for Optimal Coordination
We apply this algorithm to analyze the strategically relevant information for tasks in both a standard and a partially observable version of the Overcooked environment.
Time-Efficient Reward Learning via Visually Assisted Cluster Ranking
One of the most successful paradigms for reward learning uses human feedback in the form of comparisons.
UniMASK: Unified Inference in Sequential Decision Problems
Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks.
Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration
We show that using optimal behavior as a prior for human models makes these models vastly more data-efficient and able to generalize to new environments.
Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers
Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks.
Estimating and Penalizing Induced Preference Shifts in Recommender Systems
These steps involve two challenging ingredients: estimation requires anticipating how hypothetical algorithms would influence user preferences if deployed - we do this by using historical user interaction data to train a predictive user model which implicitly contains their preference dynamics; evaluation and optimization additionally require metrics to assess whether such influences are manipulative or otherwise unwanted - we use the notion of "safe shifts", that define a trust region within which behavior is safe: for instance, the natural way in which users would shift without interference from the system could be deemed "safe".
Evaluating the Robustness of Collaborative Agents
We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness.
On the Utility of Learning about Humans for Human-AI Coordination
While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves.
