Search Results for author:
Found 6 papers, 2 papers with code
Meta- (out-of-context) learning in neural networks
Brown et al. (2020) famously introduced the phenomenon of in-context learning in large language models (LLMs).
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Defining and Characterizing Reward Hacking
We provide the first formal definition of reward hacking, a phenomenon where optimizing an imperfect proxy reward function, $\mathcal{\tilde{R}}$, leads to poor performance according to the true reward function, $\mathcal{R}$.
Combining Reward Information from Multiple Sources
We study this problem in the setting with two conflicting reward functions learned from different sources.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Preferences Implicit in the State of the World
We find that information from the initial state can be used to infer both side effects that should be avoided as well as preferences for how the environment should be organized.
