Search Results for author:
Found 6 papers, 2 papers with code
AI Alignment with Changing and Influenceable Reward Functions
Existing AI alignment approaches assume that preferences are static, which is unrealistic: our preferences change, and may even be influenced by our interactions with AI systems themselves.
Intent Demonstration in General-Sum Dynamic Games via Iterative Linear-Quadratic Approximations
Autonomous agents should be able to coordinate with other agents without knowing their intents ahead of time.
Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
We prove that standard applications of preference learning, including reinforcement learning from human feedback (RLHF), implicitly aggregate over hidden contexts according to a well-known voting rule called Borda count.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
On the Computational Consequences of Cost Function Design in Nonlinear Optimal Control
However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error.
Analyzing Human Models that Adapt Online
This enables us to leverage tools from reachability analysis and optimal control to compute the set of hypotheses the robot could learn in finite time, as well as the worst and best-case time it takes to learn them.
