Search Results for author:
Found 28 papers, 19 papers with code
Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
Deep representation learning methods struggle with continual learning, suffering from both catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful units.
MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning
Our algorithm improves the agent's focus with useful masks, while its efficient Masker network only adds 0. 2% more parameters to the original structure, in contrast to previous work.
Elephant Neural Networks: Born to Be a Continual Learner
We show that by simply replacing classical activation functions with elephant activation functions, we can significantly improve the resilience of neural networks to catastrophic forgetting.
Maintaining Plasticity in Deep Continual Learning
If deep-learning systems are applied in a continual learning setting, then it is well known that they may fail to remember earlier examples.
Correcting discount-factor mismatch in on-policy policy gradient methods
The policy gradient theorem gives a convenient form of the policy gradient in terms of three factors: an action value, a gradient of the action likelihood, and a state distribution involving discounting called the \emph{discounted stationary distribution}.
Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo
One of the key shortcomings of existing Thompson sampling algorithms is the need to perform a Gaussian approximation of the posterior distribution, which is not a good surrogate in most practical settings.
Reducing the Cost of Cycle-Time Tuning for Real-World Policy Optimization
In this work, we investigate the widely-used baseline hyper-parameter values of two policy gradient algorithms -- PPO and SAC -- across different cycle times.
Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning
Modern representation learning methods often struggle to adapt quickly under non-stationarity because they suffer from catastrophic forgetting and decaying plasticity.
Learning to Optimize for Reinforcement Learning
Reinforcement learning (RL) is essentially different from supervised learning and in practice these learned optimizers do not work well even in simple RL tasks.
Dynamic Decision Frequency with Continuous Options
In classic reinforcement learning algorithms, agents make decisions at discrete and fixed time intervals.
HesScale: Scalable Computation of Hessian Diagonals
Second-order optimization uses curvature information about the objective function, which can help in faster convergence.
Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computers
A common setup for a robotic agent is to have two different computers simultaneously: a resource-limited local computer tethered to the robot and a powerful remote computer connected wirelessly.
Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidation
The experience replay buffer, a standard component in deep reinforcement learning, is often used to reduce forgetting and improve sample efficiency by storing experiences in a large buffer and using them for training later.
Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots
An oft-ignored challenge of real-world reinforcement learning is that the real world does not pause when agents make learning updates.
A Temporal-Difference Approach to Policy Gradient Estimation
The policy gradient theorem (Sutton et al., 2000) prescribes the usage of a cumulative discounted state distribution under the target policy to approximate the gradient.
An Alternate Policy Gradient Estimator for Softmax Policies
Policy gradient (PG) estimators are ineffective in dealing with softmax policies that are sub-optimally saturated, which refers to the situation when the policy concentrates its probability mass on sub-optimal actions.
Continual Backprop: Stochastic Gradient Descent with Persistent Randomness
The Backprop algorithm for learning in neural networks utilizes two mechanisms: first, stochastic gradient descent and second, initialization with small random weights, where the latter is essential to the effectiveness of the former.
Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergences
Approximate Policy Iteration (API) algorithms alternate between (approximate) policy evaluation and (approximate) greedification.
Model-free Policy Learning with Reward Gradients
As a key component in reinforcement learning, the reward function is usually devised carefully to guide the agent.
Autoregressive Policies for Continuous Control Deep Reinforcement Learning
We introduce a family of stationary autoregressive (AR) stochastic processes to facilitate exploration in continuous control domains.
Benchmarking Reinforcement Learning Algorithms on Real-World Robots
The research community is now able to reproduce, analyze and build quickly on these results due to open source implementations of learning algorithms and simulated benchmark tasks.
Setting up a Reinforcement Learning Task with a Real-World Robot
Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks.
On Generalized Bellman Equations and Temporal-Difference Learning
As to its soundness, using Markov chain theory, we prove the ergodicity of the joint state-trace process under nonrestrictive conditions, and we show that associated with our scheme is a generalized Bellman equation (for the policy to be evaluated) that depends on both the evolution of $\lambda$ and the unique invariant probability measure of the state-trace process.
True Online Temporal-Difference Learning
Our results suggest that the true online methods indeed dominate the regular methods.
Emphatic Temporal-Difference Learning
Emphatic algorithms are temporal-difference learning algorithms that change their effective state distribution by selectively emphasizing and de-emphasizing their updates on different time steps.
An Empirical Evaluation of True Online TD(λ)
Our results confirm the strength of true online TD({\lambda}): 1) for sparse feature vectors, the computational overhead with respect to TD({\lambda}) is minimal; for non-sparse features the computation time is at most twice that of TD({\lambda}), 2) across all domains/representations the learning speed of true online TD({\lambda}) is often better, but never worse than that of TD({\lambda}), and 3) true online TD({\lambda}) is easier to use, because it does not require choosing between trace types, and it is generally more stable with respect to the step-size.
An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learning
In this paper we introduce the idea of improving the performance of parametric temporal-difference (TD) learning algorithms by selectively emphasizing or de-emphasizing their updates on different time steps.
Weighted importance sampling for off-policy learning with linear function approximation
Second, we show that these benefits extend to a new weighted-importance-sampling version of off-policy LSTD(lambda).
