Search Results for author:
Found 35 papers, 9 papers with code
A distributional view on multi objective policy optimization
Many real-world problems require trading off multiple competing objectives.
Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domains
We conduct a set of experiments in the RGB-stacking environment, showing that planning with the learned skills and the associated model can enable zero-shot generalization to new tasks, and can further speed up training of policies via reinforcement learning.
SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Exploration
The ability to effectively reuse prior knowledge is a key requirement when building general and flexible Reinforcement Learning (RL) agents.
How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
Our analysis, both in simulation and in the real world, shows that our approach is the best across data budgets, while standard offline RL from teacher rollouts is surprisingly effective when enough data is given.
Revisiting Gaussian mixture critics in off-policy reinforcement learning: a sample-based approach
Actor-critic algorithms that make use of distributional policy evaluation have frequently been shown to outperform their non-distributional counterparts on many challenging control tasks.
Forgetting and Imbalance in Robot Lifelong Learning with Off-policy Data
We propose the Offline Distillation Pipeline to break this trade-off by separating the training procedure into an online interaction phase and an offline distillation phase. Second, we find that training with the imbalanced off-policy data from multiple environments across the lifetime creates a significant performance drop.
Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes
We study the problem of robotic stacking with objects of complex geometry.
Ranked #2 on
Skill Generalization
on RGB-Stacking
Evaluating model-based planning and planner amortization for continuous control
There is a widespread intuition that model-based control methods should be able to surpass the data efficiency of model-free approaches.
On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning
Many advances that have improved the robustness and efficiency of deep reinforcement learning (RL) algorithms can, in one way or another, be understood as introducing additional objectives or constraints in the policy optimization step.
On Multi-objective Policy Optimization as a Tool for Reinforcement Learning
We highlight its strengths on standard MO benchmark problems and consider case studies in which we recast offline RL and learning from experts as MO problems.
From Motor Control to Team Play in Simulated Humanoid Football
In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds.
Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learning
This causes BBE to be actively detrimental to policy learning in many control tasks.
Explicit Pareto Front Optimization for Constrained Reinforcement Learning
We propose a framework that uses a multi-objective RL algorithm to find a Pareto front of policies that trades off between the reward and constraint(s), and simultaneously searches along this front for constraint-satisfying policies.
Local Search for Policy Iteration in Continuous Control
We demonstrate that additional computation spent on model-based policy improvement during learning can improve data efficiency, and confirm that model-based policy improvement during action selection can also be beneficial.
Data-efficient Hindsight Off-policy Option Learning
We introduce Hindsight Off-policy Options (HO2), a data-efficient option learning algorithm.
Acme: A Research Framework for Distributed Reinforcement Learning
These implementations serve both as a validation of our design decisions as well as an important contribution to reproducibility in RL research.
A Distributional View on Multi-Objective Policy Optimization
Many real-world problems require trading off multiple competing objectives.
Keep Doing What Worked: Behavior Modelling Priors for Offline Reinforcement Learning
In practice, however, standard off-policy algorithms fail in the batch setting for continuous control.
Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning
In practice, however, standard off-policy algorithms fail in the batch setting for continuous control.
Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
In contrast, we propose to treat hybrid problems in their 'native' form by solving them with hybrid reinforcement learning, which optimizes for discrete and continuous actions simultaneously.
Quinoa: a Q-function You Infer Normalized Over Actions
We present an algorithm for learning an approximate action-value soft Q-function in the relative entropy regularised reinforcement learning setting, for which an optimal improved policy can be recovered in closed form.
Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer
Learning robotic control policies in the real world gives rise to challenges in data efficiency, safety, and controlling the initial condition of the system.
Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models
Humans are masters at quickly learning many complex tasks, relying on an approximate understanding of the dynamics of their environments.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
+2
Augmenting learning using symmetry in a biologically-inspired domain
Invariances to translation, rotation and other spatial transformations are a hallmark of the laws of motion, and have widespread use in the natural sciences to reduce the dimensionality of systems of equations.
V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control
Some of the most successful applications of deep reinforcement learning to challenging domains in discrete and continuous control have used policy gradient methods in the on-policy setting.
Compositional Transfer in Hierarchical Reinforcement Learning
The successful application of general reinforcement learning algorithms to real-world robotics applications is often limited by their high data requirements.
General Reinforcement Learning
Hierarchical Reinforcement Learning
+4
Robust Reinforcement Learning for Continuous Control with Model Misspecification
We provide a framework for incorporating robustness -- to perturbations in the transition dynamics which we refer to as model misspecification -- into continuous control Reinforcement Learning (RL) algorithms.
Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cup
We present a method for fast training of vision based control policies on real robots.
Relative Entropy Regularized Policy Iteration
Our algorithm draws on connections to existing literature on black-box optimization and 'RL as an inference' and it can be seen either as an extension of the Maximum a Posteriori Policy Optimisation algorithm (MPO) [Abdolmaleki et al., 2018a], or as an extension of Trust Region Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) [Abdolmaleki et al., 2017b; Hansen et al., 1997] to a policy iteration scheme.
Success at any cost: value constrained model-free continuous control
We demonstrate the efficiency of our approach using a number of continuous control benchmark tasks as well as a realistic, energy-optimized quadruped locomotion task.
Maximum a Posteriori Policy Optimisation
We introduce a new algorithm for reinforcement learning called Maximum aposteriori Policy Optimisation (MPO) based on coordinate ascent on a relative entropy objective.
DeepMind Control Suite
The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforcement learning agents.
Guide Actor-Critic for Continuous Control
First, we show that GAC updates the guide actor by performing second-order optimization in the action space where the curvature matrix is based on the Hessians of the critic.
Model-Free Trajectory-based Policy Optimization with Monotonic Improvement
In order to show the monotonic improvement of our algorithm, we additionally conduct a theoretical analysis of our policy update scheme to derive a lower bound of the change in policy return between successive iterations.
Model-Based Relative Entropy Stochastic Search
Stochastic search algorithms are general black-box optimizers.
