Search Results for author:
Found 52 papers, 19 papers with code
Disentangling the Causes of Plasticity Loss in Neural Networks
Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \textit{stationary} data distribution.
A Survey of Temporal Credit Assignment in Deep Reinforcement Learning
In this survey, we review the state of the art of Temporal Credit Assignment (CA) in deep RL.
A Definition of Continual Reinforcement Learning
Using this new language, we define a continual learning agent as one that can be understood as carrying out an implicit search process indefinitely, and continual reinforcement learning as the setting in which the best agents are all continual learning agents.
On the Convergence of Bounded Agents
Standard models of the reinforcement learning problem give rise to a straightforward definition of convergence: An agent converges when its behavior or performance in each environment state stops changing.
Exploration via Epistemic Value Estimation
We propose epistemic value estimation (EVE): a recipe that is compatible with sequential decision making and with neural network function approximators.
Learning How to Infer Partial MDPs for In-Context Adaptation and Exploration
To generalize across tasks, an agent should acquire knowledge from past tasks that facilitate adaptation and exploration in future tasks.
Human-level Atari 200x faster
The task of building general agents that perform well over a wide range of tasks has been an importantgoal in reinforcement learning since its inception.
Selective Credit Assignment
Efficient credit assignment is essential for reinforcement learning algorithms in both prediction and control settings.
Chaining Value Functions for Off-Policy Learning
To accumulate knowledge and improve its policy of behaviour, a reinforcement learning agent can learn `off-policy' about policies that differ from the policy used to generate its experience.
Self-Consistent Models and Values
Learned models of the environment provide reinforcement learning (RL) agents with flexible ways of making predictions about the environment.
Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity
Strategic diversity is often essential in games: in multi-player games, for example, evaluating a player against a diverse set of strategies will yield a more accurate estimate of its performance.
Learning by Directional Gradient Descent
How should state be constructed from a sequence of observations, so as to best achieve some objective?
Introducing Symmetries to Black Box Meta Reinforcement Learning
We show that a recent successful meta RL approach that meta-learns an objective for backpropagation-based learning exhibits certain symmetries (specifically the reuse of the learning rule, and invariance to input and output permutations) that are not present in typical black-box meta RL systems.
Bootstrapped Meta-Learning
We achieve a new state-of-the art for model-free agents on the Atari ALE benchmark and demonstrate that it yields both performance and efficiency gains in multi-task meta-learning.
Learning Expected Emphatic Traces for Deep RL
We develop a multi-step emphatic weighting that can be combined with replay, and a time-reversed $n$-step TD learning algorithm to learn the required emphatic weighting.
Emphatic Algorithms for Deep Reinforcement Learning
In this paper, we extend the use of emphatic methods to deep reinforcement learning agents.
Muesli: Combining Improvements in Policy Optimization
We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss.
Ranked #8 on
Atari Games
on atari game
Podracer architectures for scalable Reinforcement Learning
Supporting state-of-the-art AI research requires balancing rapid prototyping, ease of use, and quick iteration, with the ability to deploy experiments at a scale traditionally associated with production systems. Deep learning frameworks such as TensorFlow, PyTorch and JAX allow users to transparently make use of accelerators, such as TPUs and GPUs, to offload the more computationally intensive parts of training and inference in modern deep learning systems.
Synthetic Returns for Long-Term Credit Assignment
We propose state-associative (SA) learning, where the agent learns associations between states and arbitrarily distant future rewards, then propagates credit directly between the two.
Discovery of Options via Meta-Learned Subgoals
Temporal abstractions in the form of options have been shown to help reinforcement learning (RL) agents learn faster.
Forethought and Hindsight in Credit Assignment
We address the problem of credit assignment in reinforcement learning and explore fundamental questions regarding the way in which an agent can best use additional computation to propagate new information, by planning with internal models of the world to improve its predictions.
Discovering Reinforcement Learning Algorithms
Automating the discovery of update rules from data could lead to more efficient algorithms, or algorithms that are better adapted to specific environments.
Meta-Gradient Reinforcement Learning with an Objective Discovered Online
In this work, we propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network, solely from interactive experience with its environment.
Expected Eligibility Traces
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in reinforcement learning and artificial intelligence.
A Self-Tuning Actor-Critic Algorithm
Reinforcement learning algorithms are highly sensitive to the choice of hyperparameters, typically requiring significant manual effort to identify hyperparameters that perform well on a new domain.
What Can Learned Intrinsic Rewards Capture?
Furthermore, we show that unlike policy transfer methods that capture "how" the agent should behave, the learned reward functions can generalise to other kinds of agents and to changes in the dynamics of the environment by capturing "what" the agent should strive to do.
Hindsight Credit Assignment
We consider the problem of efficient credit assignment in reinforcement learning.
Conditional Importance Sampling for Off-Policy Learning
We theoretically analyse this space, and concretely investigate several algorithms that arise from this framework.
Discovery of Useful Questions as Auxiliary Tasks
Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this departs from the focus in much of machine learning on agents learning answers to externally defined questions.
Behaviour Suite for Reinforcement Learning
bsuite is a collection of carefully-designed experiments that investigate core capabilities of reinforcement learning (RL) agents with two objectives.
On Inductive Biases in Deep Reinforcement Learning
These inductive biases can take many forms, including domain knowledge and pretuned hyper-parameters.
When to use parametric models in reinforcement learning?
We examine the question of when and how parametric models are most useful in reinforcement learning.
Meta-learning of Sequential Strategies
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class.
Towards Consistent Performance on Atari using Expert Demonstrations
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games.
Universal Successor Features Approximators
We focus on one aspect in particular, namely the ability to generalise to unseen tasks.
Deep Reinforcement Learning and the Deadly Triad
In this work, we investigate the impact of the deadly triad in practice, in the context of a family of popular deep reinforcement learning models - deep Q-networks trained with experience replay - analysing how the components of this system play a role in the emergence of the deadly triad, and in the agent's performance
The Barbados 2018 List of Open Issues in Continual Learning
We want to make progress toward artificial general intelligence, namely general-purpose agents that autonomously learn how to competently act in complex environments.
Multi-task Deep Reinforcement Learning with PopArt
This means the learning algorithm is general, but each solution is not; each agent can only solve the one task it was trained on.
Ranked #1 on
Visual Navigation
on Dmlab-30
Observe and Look Further: Achieving Consistent Performance on Atari
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games.
Meta-Gradient Reinforcement Learning
Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function.
Distributed Prioritized Experience Replay
We propose a distributed architecture for deep reinforcement learning at scale, that enables agents to learn effectively from orders of magnitude more data than previously possible.
Ranked #1 on
Atari Games
on Atari 2600 Boxing
Unicorn: Continual Learning with a Universal, Off-policy Agent
Some real-world domains are best characterized as a single task, but for others this perspective is limiting.
Rainbow: Combining Improvements in Deep Reinforcement Learning
The deep reinforcement learning community has made several independent improvements to the DQN algorithm.
Ranked #9 on
Atari Games
on atari game
StarCraft II: A New Challenge for Reinforcement Learning
Finally, we present initial baseline results for canonical deep reinforcement learning agents applied to the StarCraft II domain.
Ranked #1 on
Starcraft II
on MoveToBeacon
The Predictron: End-To-End Learning and Planning
One of the key challenges of artificial intelligence is to learn models that are effective in the context of planning.
Successor Features for Transfer in Reinforcement Learning
Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks.
Learning values across many orders of magnitude
Most learning algorithms are not invariant to the scale of the function that is being approximated.
Ranked #12 on
Atari Games
on Atari 2600 Centipede
Deep Reinforcement Learning in Large Discrete Action Spaces
Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems.
Dueling Network Architectures for Deep Reinforcement Learning
In recent years there have been many successes of using deep representations in reinforcement learning.
Ranked #1 on
Atari Games
on Atari 2600 Pong
Deep Reinforcement Learning with Double Q-learning
The popular Q-learning algorithm is known to overestimate action values under certain conditions.
Ranked #10 on
Atari Games
on Atari 2600 Gopher
Learning to Predict Independent of Span
If predictions are made at a high rate or span over a large amount of time, substantial computation can be required to store all relevant observations and to update all predictions when the outcome is finally observed.
