Search Results for author:
Found 56 papers, 12 papers with code
A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning
The performance of modern reinforcement learning algorithms critically relies on tuning ever-increasing numbers of hyperparameters.
Real-Time Recurrent Learning using Trace Units in Reinforcement Learning
Recurrent Neural Networks (RNNs) are used to learn representations in partially observable environments.
The Cross-environment Hyperparameter Setting Benchmark for Reinforcement Learning
This paper introduces a new empirical methodology, the Cross-environment Hyperparameter Setting Benchmark, that compares RL algorithms across environments using a single hyperparameter setting, encouraging algorithmic development which is insensitive to hyperparameters.
Investigating the Interplay of Prioritized Replay and Generalization
One exception is Prioritized Experience Replay (PER), where sampling is done proportionally to TD errors, inspired by the success of prioritized sweeping in dynamic programming.
Position: Benchmarking is Limited in Reinforcement Learning Research
Novel reinforcement learning algorithms, or improvements on existing ones, are commonly justified by evaluating their performance on benchmark environments and are compared to an ever-changing set of standard algorithms.
A New View on Planning in Online Reinforcement Learning
In this paper, we avoid this limitation by constraining background planning to a set of (abstract) subgoals and learning only local, subgoal-conditioned models.
Model-based Reinforcement Learning
reinforcement-learning
+1
K-percent Evaluation for Lifelong RL
In continual or lifelong reinforcement learning, access to the environment should be limited.
Application-Driven Innovation in Machine Learning
As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly important.
GVFs in the Real World: Making Predictions Online for Water Treatment
In this paper we investigate the use of reinforcement-learning based prediction approaches for a real drinking-water treatment plant.
Harnessing Discrete Representations For Continual Reinforcement Learning
In this work, we provide a thorough empirical investigation of the advantages of representing observations as vectors of categorical values within the context of reinforcement learning.
Predicting recovery following stroke: deep learning, multimodal data and feature selection using explainable AI
The highest classification accuracy 0. 854 was observed when 8 regions-of-interest was extracted from each MRI scan and combined with lesion size, initial severity and recovery time in a 2D Residual Neural Network. Our findings demonstrate how imaging and tabular data can be combined for high post-stroke classification accuracy, even when the dataset is small in machine learning terms.
AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning
In this paper, we introduce recurrent alternatives to the transformer self-attention mechanism that offer context-independent inference cost, leverage long-range dependencies effectively, and performs well in online reinforcement learning task.
Measuring and Mitigating Interference in Reinforcement Learning
Lastly, we outline a class of algorithms which we call online-aware that are designed to mitigate interference, and show they do reduce interference according to our measure and that they improve stability and performance in several classic control environments.
Empirical Design in Reinforcement Learning
The objective of this document is to provide answers on how we can use our unprecedented compute to do good science in reinforcement learning, as well as stay alert to potential pitfalls in our empirical design.
Loss of Plasticity in Continual Deep Reinforcement Learning
The ability to learn continually is essential in a complex and changing world.
The In-Sample Softmax for Offline Reinforcement Learning
We highlight a simple fact: it is more straightforward to approximate an in-sample \emph{softmax} using only actions in the dataset.
Agent-State Construction with Auxiliary Inputs
Finally, we show that this approach is complementary to state-of-the-art methods such as recurrent neural networks and truncated back-propagation through time, and acts as a heuristic that facilitates longer temporal credit assignment, leading to better performance.
Auxiliary task discovery through generate-and-test
In this paper, we explore an approach to auxiliary task discovery in reinforcement learning based on ideas from representation learning.
Goal-Space Planning with Subgoal Models
In this paper, we avoid this limitation by constraining background planning to a set of (abstract) subgoals and learning only local, subgoal-conditioned models.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
The performance of reinforcement learning (RL) agents is sensitive to the choice of hyperparameters.
What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy
Our focus on the role of the target policy of the auxiliary tasks is motivated by the fact that the target policy determines the behavior about which the agent wants to make a prediction and the state-action distribution that the agent is trained on, which further affects the main task learning.
Investigating the Properties of Neural Network Representations in Reinforcement Learning
In this paper we investigate the properties of representations learned by deep reinforcement learning systems.
The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents
Our results showcase the speed of learning for Pavlovian signalling, the impact that different temporal representations do (and do not) have on agent-agent coordination, and how temporal aliasing impacts agent-agent and human-agent interactions differently.
Continual Auxiliary Task Learning
Learning auxiliary tasks, such as multiple predictions about the world, can provide many benefits to reinforcement learning systems.
Reward-Respecting Subtasks for Model-Based Reinforcement Learning
Each subtask is solved to produce an option, and then a model of the option is learned and made available to the planning process.
Model-based Reinforcement Learning
reinforcement-learning
+2
Pavlovian Signalling with General Value Functions in Agent-Agent Temporal Decision Making
We further show how to computationally build this adaptive signalling process out of a fixed signalling process, characterized by fast continual prediction learning and minimal constraints on the nature of the agent receiving signals.
Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study
Additionally, we compare two different agent architectures to assess how representational choices in agent design affect the human-agent interaction.
Counterfactual Instances Explain Little
We will further illustrate how explainable AI methods that provide both causal equations and counterfactual instances can successfully explain machine learning predictions.
Learning Expected Emphatic Traces for Deep RL
We develop a multi-step emphatic weighting that can be combined with replay, and a time-reversed $n$-step TD learning algorithm to learn the required emphatic weighting.
Contrastive Counterfactual Visual Explanations With Overdetermination
A novel explainable AI method called CLEAR Image is introduced in this paper.
Emphatic Algorithms for Deep Reinforcement Learning
In this paper, we extend the use of emphatic methods to deep reinforcement learning agents.
A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning
Many algorithms have been developed for off-policy value estimation based on the linear mean squared projected Bellman error (MSPBE) and are sound under linear function approximation.
From Eye-blinks to State Construction: Diagnostic Benchmarks for Online Representation Learning
We present three new diagnostic prediction problems inspired by classical-conditioning experiments to facilitate research in online prediction learning.
Towards a practical measure of interference for reinforcement learning
In this work, we provide a definition of interference for control in reinforcement learning.
Gradient Temporal-Difference Learning with Regularized Corrections
It is still common to use Q-learning and temporal difference (TD) learning-even though they have divergence issues and sound Gradient TD alternatives exist-because divergence seems rare and they typically perform well.
Training Recurrent Neural Networks Online by Learning Explicit State Variables
Recurrent neural networks (RNNs) allow an agent to construct a state-representation from a stream of experience, which is essential in partially observable problems.
Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks
Unfortunately, the performance of deep reinforcement learning systems is sensitive to hyper-parameter settings and architecture choices.
Measurable Counterfactual Local Explanations for Any Classifier
We propose a novel method for explaining the predictions of any classifier.
Meta-descent for Online, Continual Prediction
This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems.
Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study
The question we tackle in this paper is how to sculpt the stream of experience---how to adapt the learning system's behavior---to optimize the learning of a collection of value functions.
Planning with Expectation Models
In particular, we 1) show that planning with an expectation model is equivalent to planning with a distribution model if the state value function is linear in state features, 2) analyze two common parametrization choices for approximating the expectation: linear and non-linear expectation models, 3) propose a sound model-based policy evaluation algorithm and present its convergence results, and 4) empirically demonstrate the effectiveness of the proposed planning algorithm.
The Barbados 2018 List of Open Issues in Continual Learning
We want to make progress toward artificial general intelligence, namely general-purpose agents that autonomously learn how to competently act in complex environments.
Context-Dependent Upper-Confidence Bounds for Directed Exploration
Directed exploration strategies for reinforcement learning are critical for learning an optimal policy in a minimal number of interactions with the environment.
Online Off-policy Prediction
The ability to learn behavior-contingent predictions online and off-policy has long been advocated as a key capability of predictive-knowledge learning systems but remained an open algorithmic challenge for decades.
Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement
We first provide a policy improvement result in an idealized setting, and then prove that our conditional CEM (CCEM) strategy tracks a CEM update per state, even with changing action-values.
General Value Function Networks
A general purpose strategy for state construction is to learn the state update using a Recurrent Neural Network (RNN), which updates the internal state using the current internal state and the most recent observation.
Organizing Experience: A Deeper Look at Replay Mechanisms for Sample-based Planning in Continuous State Domains
We show that a model, as opposed to a replay buffer, is particularly useful for specifying which states to sample from during planning, such as predecessor states that propagate information in reverse from a state more quickly.
Directly Estimating the Variance of the λ-Return Using Temporal-Difference Methods
This paper investigates estimating the variance of a temporal-difference learning agent's update target.
Discovery of Predictive Representations With a Network of General Value Functions
We investigate a framework for discovery: curating a large collection of predictions, which are used to construct the agent's representation of the world.
GQ($λ$) Quick Reference and Implementation Guide
This document should serve as a quick reference for and guide to the implementation of linear GQ($\lambda$), a gradient-based off-policy temporal-difference learning algorithm.
Accelerated Gradient Temporal Difference Learning
The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD({\lambda}) to data efficient least squares methods.
A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning
One of the main obstacles to broad application of reinforcement learning methods is the parameter sensitivity of our core learning algorithms.
Introspective Agents: Confidence Measures for General Value Functions
Agents of general intelligence deployed in real-world scenarios must adapt to ever-changing environmental conditions.
Investigating practical linear temporal difference learning
First, we derive two new hybrid TD policy-evaluation algorithms, which fill a gap in this collection of algorithms.
Multi-timescale Nexting in a Reinforcement Learning Robot
The term "nexting" has been used by psychologists to refer to the propensity of people and many other animals to continually predict what will happen next in an immediate, local, and personal sense.
Interval Estimation for Reinforcement-Learning Algorithms in Continuous-State Domains
The reinforcement learning community has explored many approaches to obtain- ing value estimates and models to guide decision making; these approaches, how- ever, do not usually provide a measure of confidence in the estimate.
