Search Results for author:
Found 22 papers, 6 papers with code
Capturing Individual Human Preferences with Reward Features
We argue that this is unlikely to be a good design choice in contexts with high potential for disagreement, like in the training of large language models.
Agency Is Frame-Dependent
Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence.
Optimizing Return Distributions with Distributional Dynamic Programming
To go beyond expected utilities, we combine distributional DP with stock augmentation, a technique previously introduced for classic DP in the context of risk-sensitive RL, where the MDP state is augmented with a statistic of the rewards obtained so far (since the first time step).
A Distributional Analogue to the Successor Representation
This paper contributes a new approach for distributional reinforcement learning which elucidates a clean separation of transition structure and reward in the learning process.
Distributional Reinforcement Learning
Model-based Reinforcement Learning
+2
A Definition of Continual Reinforcement Learning
Using this new language, we define a continual learning agent as one that can be understood as carrying out an implicit search process indefinitely, and continual reinforcement learning as the setting in which the best agents are all continual learning agents.
On the Convergence of Bounded Agents
Standard models of the reinforcement learning problem give rise to a straightforward definition of convergence: An agent converges when its behavior or performance in each environment state stops changing.
Generalised Policy Improvement with Geometric Policy Composition
We introduce a method for policy improvement that interpolates between the greedy approach of value-based reinforcement learning (RL) and the full planning approach typical of model-based RL.
The Phenomenon of Policy Churn
We identify and study the phenomenon of policy churn, that is, the rapid change of the greedy policy in value-based reinforcement learning.
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Unlike prior work which estimates uncertainty by training an ensemble of many models and/or value functions, this approach requires only the single model and value function which are already being learned in most model-based reinforcement learning algorithms.
Model-based Reinforcement Learning
Rolling Shutter Correction
The Option Keyboard: Combining Skills in Reinforcement Learning
Building on this insight and on previous results on transfer learning, we show how to approximate options whose cumulants are linear combinations of the cumulants of known options.
Proper Value Equivalence
The value-equivalence (VE) principle proposes a simple answer to this question: a model should capture the aspects of the environment that are relevant for value-based planning.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Risk-Aware Transfer in Reinforcement Learning using Successor Features
Sample efficiency and risk-awareness are central to the development of practical reinforcement learning (RL) for complex decision-making.
The Value Equivalence Principle for Model-Based Reinforcement Learning
As our main contribution, we introduce the principle of value equivalence: two models are value equivalent with respect to a set of functions and policies if they yield the same Bellman updates.
Model-based Reinforcement Learning
reinforcement-learning
+3
Fast reinforcement learning with generalized policy updates
Both strategies considerably reduce the amount of data needed to solve a reinforcement-learning problem.
Expected Eligibility Traces
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in reinforcement learning and artificial intelligence.
The Value-Improvement Path: Towards Better Representations for Reinforcement Learning
To test our hypothesis empirically, we augmented a standard deep RL agent with an auxiliary task of learning the value-improvement path.
Temporally-Extended ε-Greedy Exploration
Recent work on exploration in reinforcement learning (RL) has led to a series of increasingly complex solutions to the problem.
Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
In this paper we extend the SFs & GPI framework in two ways.
Universal Successor Features Approximators
We focus on one aspect in particular, namely the ability to generalise to unseen tasks.
Fast deep reinforcement learning using online adjustments from the past
We propose Ephemeral Value Adjusments (EVA): a means of allowing deep reinforcement learning agents to rapidly adapt to experience in their replay buffer.
Unicorn: Continual Learning with a Universal, Off-policy Agent
Some real-world domains are best characterized as a single task, but for others this perspective is limiting.
Successor Features for Transfer in Reinforcement Learning
Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks.
