Search Results for author:
Found 491 papers, 228 papers with code
Feature Construction for Inverse Reinforcement Learning
The goal of inverse reinforcement learning is to find a reward function for a Markov decision process, given example traces from its optimal policy.
Nonlinear Inverse Reinforcement Learning with Gaussian Processes
We present a probabilistic algorithm for nonlinear inverse reinforcement learning.
Exploring Deep and Recurrent Architectures for Optimal Control
In this paper, we explore the application of deep and recurrent neural networks to a continuous, high-dimensional locomotion task, where the network is used to represent a control policy that maps the state of the system (represented by joint angles) directly to the torques at each joint.
Variational Policy Search via Trajectory Optimization
In order to learn effective control policies for dynamical systems, policy search methods must be able to discover successful executions of the desired task.
Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics
We present a policy search method that uses iteratively refitted local linear models to optimize trajectory distributions for large, continuous problems.
Learning Contact-Rich Manipulation Skills with Guided Policy Search
Autonomous learning of object manipulation skills can enable robots to acquire rich behavioral repertoires that scale to the variety of objects found in the real world.
Robotics
Trust Region Policy Optimization
We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement.
End-to-End Training of Deep Visuomotor Policies
Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control.
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Policy gradient methods are an appealing approach in reinforcement learning because they directly optimize the cumulative reward and can straightforwardly be used with nonlinear function approximators such as neural networks.
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
By parameterizing our learned model with a neural network, we are able to develop a scalable and efficient approach to exploration bonuses that can be applied to tasks with complex, high-dimensional state spaces.
Ranked #24 on
Atari Games
on Atari 2600 Q*Bert
Learning Deep Neural Network Policies with Continuous Memory States
We evaluate our method on tasks involving continuous control in manipulation and navigation settings, and show that our method can learn complex policies that successfully complete a range of tasks that require memory.
Recurrent Network Models for Human Dynamics
We propose the Encoder-Recurrent-Decoder (ERD) model for recognition and prediction of human body pose in videos and motion capture.
Ranked #8 on
Human Pose Forecasting
on Human3.6M
(MAR, walking, 1,000ms metric)
Deep Spatial Autoencoders for Visuomotor Learning
Our method uses a deep spatial autoencoder to acquire a set of feature points that describe the environment for the current task, such as the positions of objects, and then learns a motion skill with these feature points using an efficient reinforcement learning method based on local linear models.
Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search
We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment.
One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors
One of the key challenges in applying reinforcement learning to complex robotic control tasks is the need to gather large amounts of experience in order to find an effective policy for the task at hand.
Model-based Reinforcement Learning
Model Predictive Control
+3
Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration
In this paper, we present a robotic model-based reinforcement learning method that combines ideas from model identification and model predictive control.
Model-based Reinforcement Learning
Model Predictive Control
+2
MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Deep neural networks are powerful parametric models that can be trained efficiently using the backpropagation algorithm.
Adapting Deep Visuomotor Representations with Weak Pairwise Constraints
We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains.
Learning Visual Predictive Models of Physics for Playing Billiards
The ability to plan and execute goal specific actions in varied, unexpected settings is a central requirement of intelligent agents.
Value Iteration Networks
We introduce the value iteration network (VIN): a fully differentiable neural network with a `planning module' embedded within.
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
We explore how inverse optimal control (IOC) can be used to learn behaviors from demonstrations, with applications to torque control of high-dimensional robotic systems.
Continuous Deep Q-Learning with Model-based Acceleration
In this paper, we explore algorithms and representations to reduce the sample complexity of deep reinforcement learning for continuous control tasks.
PLATO: Policy Learning using Adaptive Trajectory Optimization
PLATO also maintains the MPC cost as an objective to avoid highly undesirable actions that would result from strictly following the learned policy before it has been fully trained.
Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection
We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images.
Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks.
Backprop KF: Learning Discriminative Deterministic State Estimators
We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks.
Unsupervised Learning for Physical Interaction through Video Prediction
A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment.
Ranked #26 on
Video Generation
on BAIR Robot Pushing
Learning to Poke by Poking: Experiential Learning of Intuitive Physics
We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics.
Guided Policy Search as Approximate Mirror Descent
Guided policy search algorithms can be used to optimize complex nonlinear policies, such as deep neural networks, without directly computing policy gradients in the high-dimensional parameter space.
Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer
Using deep reinforcement learning to train general purpose neural network policies alleviates some of the burden of manual representation engineering by using expressive policy classes, but exacerbates the challenge of data collection, since such methods tend to be less efficient than RL with low-dimensional, hand-designed representations.
Learning from the Hindsight Plan -- Episodic MPC Improvement
To bring the next real-world execution closer to the hindsight plan, our approach learns to re-shape the original cost function with the goal of satisfying the following property: short horizon planning (as realistic during real executions) with respect to the shaped cost should result in mimicking the hindsight plan.
Deep Reinforcement Learning for Tensegrity Robot Locomotion
We evaluate our method with real-world and simulated experiments on the SUPERball tensegrity robot, showing that the learned policies generalize to changes in system parameters, unreliable sensor measurements, and variation in environmental conditions, including varied terrains and a range of different gravities.
Path Integral Guided Policy Search
We extend GPS in the following ways: (1) we propose the use of a model-free local optimizer based on path integral stochastic optimal control (PI2), which enables us to learn local policies for tasks with highly discontinuous contact dynamics; and (2) we enable GPS to train on a new set of task instances in every iteration by using on-policy sampling: this increases the diversity of the instances that the policy is trained on, and is crucial for achieving good generalization.
Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search
In this work, we explore distributed and asynchronous policy learning as a means to achieve generalization and improved training times on challenging, real-world manipulation tasks.
Deep Visual Foresight for Planning Robot Motion
A key challenge in scaling up robot learning to many skills and environments is removing the need for human supervision, so that robots can collect their own data and improve their own performance without being limited by the cost of requesting human feedback.
Model-based Reinforcement Learning
Model Predictive Control
+2
Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
In this paper, we demonstrate that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots.
Reset-Free Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States
Autonomous learning of robotic skills can allow general-purpose robots to learn wide behavioral repertoires without requiring extensive manual engineering.
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
Sample complexity and safety are major challenges when learning policies with reinforcement learning for real-world tasks, especially when the policies are represented using rich function approximators like deep neural networks.
Modular Multitask Reinforcement Learning with Policy Sketches
We describe a framework for multitask deep reinforcement learning guided by policy sketches.
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
We analyze the connection between Q-Prop and existing model-free algorithms, and use control variate theory to derive two variants of Q-Prop with conservative and aggressive adaptation.
A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
In particular, we demonstrate an equivalence between a sample-based algorithm for maximum entropy IRL and a GAN in which the generator's density can be evaluated and is provided as an additional input to the discriminator.
CAD2RL: Real Single-Image Flight without a Single Real Image
We propose a learning method that we call CAD$^2$RL, which can be used to perform collision-free indoor flight in the real world while being trained entirely on 3D CAD models.
Learning Dexterous Manipulation Policies from Experience and Imitation
We demonstrate that such controllers can perform the task robustly, both in simulation and on the physical platform, for a limited range of initial conditions around the trained starting state.
Guided Policy Search via Approximate Mirror Descent
Guided policy search algorithms can be used to optimize complex nonlinear policies, such as deep neural networks, without directly computing policy gradients in the high-dimensional parameter space.
Generalizing Skills with Semi-Supervised Reinforcement Learning
We evaluate our method on challenging tasks that require control directly from images, and show that our approach can improve the generalization of a learned deep neural network policy by using experience for which no reward function is available.
Unsupervised Perceptual Rewards for Imitation Learning
We present a method that is able to identify key intermediate steps of a task from only a handful of demonstration sequences, and automatically identify the most discriminative features for identifying these steps.
Uncertainty-Aware Reinforcement Learning for Collision Avoidance
However, practical deployment of reinforcement learning methods must contend with the fact that the training process itself can be unsafe for the robot.
Cognitive Mapping and Planning for Visual Navigation
The accumulated belief of the world enables the agent to track visited regions of the environment.
Reinforcement Learning with Deep Energy-Based Policies
We propose a method for learning expressive energy-based policies for continuous states and actions, which has been feasible only in tabular domains before.
EX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Deep reinforcement learning algorithms have been shown to learn complex tasks using highly general policy classes.
Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics.
Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
People can learn a wide range of tasks from their own experience, but can also learn from observing other creatures.
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning.
Goal-Driven Dynamics Learning via Bayesian Optimization
Real-world robots are becoming increasingly complex and commonly act in poorly understood environments where it is extremely challenging to model or learn their true dynamics.
Learning Visual Servoing with Deep Features and Fitted Q-Iteration
Our approach is based on servoing the camera in the space of learned visual features, rather than image pixels or manually-designed keypoints.
Time-Contrastive Networks: Self-Supervised Learning from Video
While representations are learned from an unlabeled collection of task-related videos, robot behaviors such as pouring are learned by watching a single 3rd-person demonstration by a human.
Ranked #3 on
Video Alignment
on UPenn Action
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Off-policy model-free deep reinforcement learning methods using previously collected data can improve sample efficiency over on-policy policy gradient techniques.
End-to-End Learning of Semantic Grasping
We consider the task of semantic robotic grasping, in which a robot picks up an object of a user-specified class using only monocular images.
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration
We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation.
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
Imitation learning is an effective approach for autonomous systems to acquire control policies when an explicit reward function is unavailable, using supervision provided as demonstrations from an expert, typically a human operator.
GPLAC: Generalizing Vision-Based Robotic Skills using Weakly Labeled Images
We show that pairing interaction data from just a single environment with a diverse dataset of weakly labeled data results in greatly improved generalization to unseen environments, and show that this generalization depends on both the auxiliary objective and the attentional architecture that we propose.
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
Model-free deep reinforcement learning algorithms have been shown to be capable of learning a wide range of robotic skills, but typically require a very large number of samples to achieve good performance.
Model-based Reinforcement Learning
Model Predictive Control
+2
Deep Object-Centric Representations for Generalizable Robot Learning
We devise an object-level attentional mechanism that can be used to determine relevant objects from a few trajectories or demonstrations, and then immediately incorporate those objects into a learned policy.
MBMF: Model-Based Priors for Model-Free Reinforcement Learning
Reinforcement Learning is divided in two main paradigms: model-free and model-based.
One-Shot Visual Imitation Learning via Meta-Learning
In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration.
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
We extensively evaluate our approaches with a total of more than 25, 000 physical test grasps, studying a range of simulation conditions and domain adaptation methods, including a novel extension of pixel-level domain adaptation that we term the GraspGAN.
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency.
Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation
To address the need to learn complex policies with few samples, we propose a generalized computation graph that subsumes value-based model-free methods and model-based methods, with specific instantiations interpolating between model-free and model-based.
Self-Supervised Visual Planning with Temporal Skip Connections
One learning signal that is always available for autonomously collected data is prediction: if a robot can learn to predict the future, it can use this predictive model to take actions to produce desired outcomes, such as moving an object to a particular location.
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch.
Stochastic Variational Video Prediction
We find that our proposed method produces substantially improved video predictions when compared to the same model without stochasticity, and to other stochastic video prediction methods.
Ranked #5 on
Video Prediction
on KTH
Learning Robust Rewards with Adversarial Inverse Reinforcement Learning
Reinforcement learning provides a powerful and general framework for decision making and control, but its application in practice is often hindered by the need for extensive feature and reward engineering.
Ranked #3 on
MuJoCo Games
on Ant
Regret Minimization for Partially Observable Deep Reinforcement Learning
Deep reinforcement learning algorithms that estimate state and state-action value functions have been shown to be effective in a variety of challenging domains, including learning control strategies from raw image pixels.
Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
Learning to learn is a powerful paradigm for enabling models to learn from data more effectively and efficiently.
Learning with Latent Language
The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world.
Learning Image-Conditioned Dynamics Models for Control of Under-actuated Legged Millirobots
We present an approach for controlling a real-world legged millirobot that is based on learned neural network models.
Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning
In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and reset policy, with the reset policy resetting the environment for a subsequent attempt.
Divide-and-Conquer Reinforcement Learning
In this paper, we develop a novel algorithm that instead partitions the initial state space into "slices", and optimizes an ensemble of policies, each on a different slice.
Sim2Real View Invariant Visual Servoing by Recurrent Control
To this end, we train a deep recurrent controller that can automatically determine which actions move the end-point of a robotic arm to a desired object.
Unifying Map and Landmark Based Representations for Visual Navigation
This works presents a formulation for visual navigation that unifies map based spatial reasoning and path planning, with landmark based robust plan execution in noisy environments.
Self-Supervised Learning of Object Motion Through Adversarial Video Prediction
In this paper, we study the problem of multi-step video prediction, where the goal is to predict a sequence of future frames conditioned on a short context.
Learning Robust Rewards with Adverserial Inverse Reinforcement Learning
Reinforcement learning provides a powerful and general framework for decision making and control, but its application in practice is often hindered by the need for extensive feature and reward engineering.
Combining Model-based and Model-free RL via Multi-step Control Variates
Model-free deep reinforcement learning algorithms are able to successfully solve a wide range of continuous control tasks, but typically require many on-policy samples to achieve good performance.
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
A platform for Applied Reinforcement Learning (Applied RL)
Ranked #1 on
Continuous Control
on Lunar Lander (OpenAI Gym)
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task.
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
Humans and animals are capable of learning a new behavior by observing others perform the skill just once.
Shared Autonomy via Deep Reinforcement Learning
In shared autonomy, user input is combined with semi-autonomous control to achieve a common goal.
Reinforcement Learning from Imperfect Demonstrations
We propose a unified reinforcement learning algorithm, Normalized Actor-Critic (NAC), that effectively normalizes the Q-function, reducing the Q-values of actions unseen in the demonstration data.
Diversity is All You Need: Learning Skills without a Reward Function
On a variety of simulated robotic tasks, we show that this simple objective results in the unsupervised emergence of diverse skills, such as walking and jumping.
Meta-Reinforcement Learning of Structured Exploration Strategies
Exploration is a fundamental challenge in reinforcement learning (RL).
Temporal Difference Models: Model-Free Deep RL for Model-Based Control
TDMs combine the benefits of model-free and model-based RL: they leverage the rich information in state transitions to learn very efficiently, while still attaining asymptotic performance that exceeds that of direct model-based RL methods.
The Mirage of Action-Dependent Baselines in Reinforcement Learning
Policy gradient methods are a widely used class of model-free reinforcement learning algorithms where a state-dependent baseline is used to reduce gradient estimator variance.
Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning
By enabling wider use of learned dynamics models within a model-free reinforcement learning algorithm, we improve value estimation, which, in turn, reduces the sample complexity of learning.
Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods
In this paper, we explore deep reinforcement learning algorithms for vision-based robotic grasping.
Learning Flexible and Reusable Locomotion Primitives for a Microrobot
This approach formalizes locomotion as a contextual policy search task to collect data, and subsequently uses that data to learn multi-objective locomotion primitives that can be used for planning.
Composable Deep Reinforcement Learning for Robotic Manipulation
Second, we show that policies learned with soft Q-learning can be composed to create new policies, and that the optimality of the resulting policy can be bounded in terms of the divergence between the composed policies.
Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time.
Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning
Synthesizing physiologically-accurate human movement in a variety of conditions can help practitioners plan surgeries, design experiments, or prototype assistive devices in simulated environments, reducing time and costs and improving treatment outcomes.
Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments
In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course.
Recall Traces: Backtracking Models for Efficient Reinforcement Learning
In many environments only a tiny subset of all states yield high reward.
Universal Planning Networks
We find that the representations learned are not only effective for goal-directed visual imitation via gradient-based trajectory optimization, but can also provide a metric for specifying goals using images.
Stochastic Adversarial Video Prediction
However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
We further explore a number of methods for integrating multiple clips into the learning process to develop multi-skilled agents capable of performing a rich repertoire of diverse skills.
Latent Space Policies for Hierarchical Reinforcement Learning
In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acquires a range of diverse strategies via a maximum entropy reinforcement learning objective.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable.
Data-Efficient Hierarchical Reinforcement Learning
In this paper, we study how we can develop HRL algorithms that are general, in that they do not make onerous additional assumptions beyond standard RL algorithms, and efficient, in the sense that they can be used with modest numbers of interaction samples, making them suitable for real-world problems such as robotic control.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior
Inferring intent from observed behavior has been studied extensively within the frameworks of Bayesian inverse planning and inverse reinforcement learning.
Few-Shot Segmentation Propagation with Guided Networks
Learning-based methods for visual segmentation have made progress on particular types of segmentation tasks, but are limited by the necessary supervision, the narrow definitions of fixed tasks, and the lack of control during inference for correcting errors.
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
This model -- a deep, multimodal convolutional network -- predicts the outcome of a candidate grasp adjustment, and then executes a grasp by iteratively selecting the most promising actions.
Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition
We propose variational inverse control with events (VICE), which generalizes inverse reinforcement learning methods to cases where full demonstrations are not needed, such as when only samples of desired goal states are available.
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance.
Model-based Reinforcement Learning
reinforcement-learning
+1
Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
A significant challenge for the practical application of reinforcement learning in the real world is the need to specify an oracle reward function that correctly defines a task.
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
In robotics, this ability is referred to as visual servoing: moving a tool or end-point to a desired location using primarily visual feedback.
Probabilistic Model-Agnostic Meta-Learning
However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small dataset for a new task can simply be too ambiguous to acquire a single model (e. g., a classifier) for that task that is accurate.
Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
We show that we can learn continuous latent representations of trajectories, which are effective in solving temporally extended and multi-stage problems.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Unsupervised Meta-Learning for Reinforcement Learning
In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks.
Learning Instance Segmentation by Interaction
The agent uses its current segmentation model to infer pixels that constitute objects and refines the segmentation model by interacting with these pixels.
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach.
Universal Planning Networks: Learning Generalizable Representations for Visuomotor Control
A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization.
Visual Reinforcement Learning with Imagined Goals
For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires.
Automatically Composing Representation Transformations as a Means for Generalization
A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -- either training a separate learner per task or training a single learner for all tasks -- both have difficulty with such generalization because they do not leverage the compositional structure of the task distribution.
Time-Agnostic Prediction: Predicting Predictable Video Frames
Prediction is arguably one of the most basic functions of an intelligent system.
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
Model-based reinforcement learning (RL) has proven to be a data efficient approach for learning control tasks but is difficult to utilize in domains with complex observations such as images.
Model-based Reinforcement Learning
reinforcement-learning
+1
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework.
Unsupervised Exploration with Deep Model-Based Reinforcement Learning
We show that both challenges can be addressed by representing model-uncertainty, which can both guide exploration in the unsupervised phase and ensure that the errors in the model are not exploited by the planner in the goal-directed phase.
Model-based Reinforcement Learning
reinforcement-learning
+1
EMI: Exploration with Mutual Information Maximizing State and Action Embeddings
Policy optimization struggles when the reward feedback signal is very sparse and essentially becomes a random search algorithm until the agent stumbles upon a rewarding or the goal state.
What Would pi* Do?: Imitation Learning via Off-Policy Reinforcement Learning
Learning to imitate expert actions given demonstrations containing image observations is a difficult problem in robotic control.
Few-Shot Goal Inference for Visuomotor Learning and Planning
To that end, we formulate the few-shot objective learning problem, where the goal is to learn a task objective from only a few example images of successful end states for that task.
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients.
Time Reversal as Self-Supervision
We test our method on the domain of assembly, specifically the mating of tetris-style block pairs.
EMI: Exploration with Mutual Information
Reinforcement learning algorithms struggle when the reward signal is very sparse.
Near-Optimal Representation Learning for Hierarchical Reinforcement Learning
We study the problem of representation learning in goal-conditioned hierarchical reinforcement learning.
Unsupervised Learning via Meta-Learning
A central goal of unsupervised learning is to acquire representations from unlabeled data or experience that can be used for more effective learning of downstream tasks from modest amounts of labeled data.
Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning
We demonstrate that this idea can be combined with a video-prediction based controller to enable complex behaviors to be learned from scratch using only raw visual inputs, including grasping, repositioning objects, and non-prehensile manipulation.
SFV: Reinforcement Learning of Physical Skills from Videos
In this paper, we propose a method that enables physically simulated characters to learn skills from videos (SFV).
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Dexterous multi-fingered robotic hands can perform a wide range of manipulation skills, making them an appealing component for general-purpose robotic manipulators.
Deep Imitative Models for Flexible Inference, Planning, and Control
Yet, reward functions that evoke desirable behavior are often difficult to specify.
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation
We show that a simulated robotic car and a real-world RC car can gather data and train fully autonomously without any human-provided labels beyond those needed to train the detectors, and then at test-time be able to accomplish a variety of different tasks.
One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks
We consider the problem of learning multi-stage vision-based tasks on a real robot from a single video of a human performing the task, while leveraging demonstration data of subtasks with other objects.
Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
We formulate an arithmetic relationship between feature vectors from this observation, and use it to learn a representation of scenes and objects that can then be used to identify object instances, localize them in the scene, and perform goal-directed grasping tasks where the robot must retrieve commanded objects from a bin.
Learning Actionable Representations with Goal-Conditioned Policies
Most prior work on representation learning has focused on generative approaches, learning representations that capture all underlying factors of variation in the observation space in a more disentangled or well-ordered manner.
Guiding Policies with Language via Meta-Learning
However, a single instruction may be insufficient to fully communicate our intent or, even if it is, may be insufficient for an autonomous agent to actually understand how to perform the desired task.
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
Deep reinforcement learning (RL) algorithms can learn complex robotic skills from raw sensory inputs, but have yet to achieve the kind of broad generalization and applicability demonstrated by deep learning methods in supervised domains.
Visual Memory for Robust Path Following
Equipped with this abstraction, a second network observes the world and decides how to act to retrace the path under noisy actuation and a changing environment.
Residual Reinforcement Learning for Robot Control
In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL.
Soft Actor-Critic Algorithms and Applications
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks
Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power.
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
The goal in this paper is to develop a method for continual online learning from an incoming stream of data, using deep neural network models.
Learning to Walk via Deep Reinforcement Learning
In this paper, we propose a sample-efficient deep RL algorithm based on maximum entropy RL that requires minimal per-task tuning and only a modest number of trials to learn neural network policies.
Robustness to Out-of-Distribution Inputs via Task-Aware Generative Uncertainty
Our method estimates an uncertainty measure about the model's prediction, taking into account an explicit (generative) model of the observation distribution to handle out-of-distribution inputs.
Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
Object-based factorizations provide a useful level of abstraction for interacting with the world.
Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning
Designing effective low-level robot controllers often entail platform-specific implementations that require manual heuristic parameter tuning, significant system knowledge, or long design times.
Model-based Reinforcement Learning
reinforcement-learning
+1
InfoBot: Transfer and Exploration via the Information Bottleneck
In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
Artificial Intelligence for Prosthetics - challenge solutions
In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge, participants were tasked with building a controller for a musculoskeletal model with a goal of matching a given time-varying velocity vector.
Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight
Deep reinforcement learning provides a promising approach for vision-based control of real-world robots.
From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that language-conditioned rewards are more transferable than language-conditioned policies to new environments.
Online Meta-Learning
Meta-learning views this problem as learning a prior over model parameters that is amenable for fast adaptation on a new task, but typically assumes the set of tasks are available together as a batch.
Diagnosing Bottlenecks in Deep Q-learning Algorithms
Q-learning methods represent a commonly used class of algorithms in reinforcement learning: they are generally efficient and simple, and can be combined readily with function approximators for deep reinforcement learning (RL).
Model-Based Reinforcement Learning for Atari
We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.
Ranked #12 on
Atari Games 100k
on Atari 100k
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions.
Ranked #15 on
Video Generation
on BAIR Robot Pushing
Learning Latent Plans from Play
Learning from play (LfP) offers three main advantages: 1) It is cheap.
Robotics
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning
Autonomous agents that must exhibit flexible and broad capabilities will need to be equipped with large repertoires of skills.
Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matching
We propose a novel framing of the problem as multi-modal recognition: the goal of our system is to recognize, given a visual and tactile observation, whether or not these observations correspond to the same object.
Manipulation by Feel: Touch-Based Control with Deep Predictive Models
Touch sensing is widely acknowledged to be important for dexterous robotic manipulation, but exploiting tactile sensing for continuous, non-prehensile manipulation is challenging.
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
In our approach, we perform online probabilistic filtering of latent task variables to infer how to solve a new task from small amounts of experience.
Wasserstein Dependency Measure for Representation Learning
Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning.
Guided Meta-Policy Search
Reinforcement learning (RL) algorithms have demonstrated promising results on complex tasks, yet often require impractical numbers of samples since they learn from scratch.
Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight
Machine learning techniques have enabled robots to learn narrow, yet complex tasks and also perform broad, yet simple skills with a wide variety of objects.
End-to-End Robotic Reinforcement Learning without Reward Engineering
In this paper, we propose an approach for removing the need for manual engineering of reward specifications by enabling a robot to learn from a modest number of examples of successful outcomes, followed by actively solicited queries, where the robot shows the user a state and asks for a label to determine whether that state represents successful completion of the task.
Transfer and Exploration via the Information Bottleneck
In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
Learning Actionable Representations with Goal Conditioned Policies
Most prior work on representation learning has focused on generative approaches, learning representations that capture all the underlying factors of variation in the observation space in a more disentangled or well-ordered manner.
Meta-Learning to Guide Segmentation
To explore generalization, we analyze guidance as a bridge between different levels of supervision to segment classes as the union of instances.
Learning to Reinforcement Learn by Imitation
Meta-reinforcement learning aims to learn fast reinforcement learning (RL) procedures that can be applied to new tasks or environments.
Few-Shot Intent Inference via Meta-Inverse Reinforcement Learning
A significant challenge for the practical application of reinforcement learning toreal world problems is the need to specify an oracle reward function that correctly defines a task.
Reasoning About Physical Interactions with Object-Centric Models
Object-based factorizations provide a useful level of abstraction for interacting with the world.
PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information.
Data-efficient Learning of Morphology and Controller for a Microrobot
Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design.
REPLAB: A Reproducible Low-Cost Arm Benchmark Platform for Robotic Learning
We envision REPLAB as a framework for reproducible research across manipulation tasks, and as a step in this direction, we define a template for a grasping benchmark consisting of a task definition, evaluation protocol, performance measures, and a dataset of 92k grasp attempts.
MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies
In this work, we propose multiplicative compositional policies (MCP), a method for learning reusable motor skills that can be composed to produce a range of complex behaviors.
Adversarial Policies: Attacking Deep Reinforcement Learning
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers.
SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
Theoretically, we show that SQIL can be interpreted as a regularized variant of BC that uses a sparsity prior to encourage long-horizon imitation.
Causal Confusion in Imitation Learning
Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment.
Safety Augmented Value Estimation from Demonstrations (SAVED): Safe Deep Model-Based RL for Sparse Cost Robotic Tasks
Reinforcement learning (RL) for robotics is challenging due to the difficulty in hand-engineering a dense cost function, which can lead to unintended behavior, and dynamical uncertainty, which makes exploration and constraint satisfaction challenging.
Model-based Reinforcement Learning
reinforcement-learning
+1
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
Bootstrapping error is due to bootstrapping from actions that lie outside of the training data distribution, and it accumulates via the Bellman backup operator.
Off-Policy Evaluation via Off-Policy Classification
However, for high-dimensional observations, such as images, models of the environment can be difficult to fit and value-based methods can make IS hard to use or even ill-conditioned, especially when dealing with continuous action spaces.
Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
Imitation learning allows agents to learn complex behaviors from demonstrations.
Learning Powerful Policies by Using Consistent Dynamics Model
There is enough evidence that humans build a model of the environment, not only by observing the environment but also by interacting with the environment.
Efficient Exploration via State Marginal Matching
The SMM objective can be viewed as a two-player, zero-sum game between a state density model and a parametric policy, an idea that we use to build an algorithm for optimizing the SMM objective.
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
We introduce a general control algorithm that combines the strengths of planning and reinforcement learning to effectively solve these tasks.
Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction.
When to Trust Your Model: Model-Based Policy Optimization
Designing effective model-based reinforcement learning algorithms is difficult because the ease of data generation must be weighed against the bias of model-generated data.
Model-based Reinforcement Learning
reinforcement-learning
+1
Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives
Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
Deep reinforcement learning (RL) algorithms can use high-capacity deep networks to learn directly from image observations.
Dynamics-Aware Unsupervised Discovery of Skills
Conventionally, model-based reinforcement learning (MBRL) aims to learn a global model for the dynamics of the environment.
Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
We show that dynamical distances can be used in a semi-supervised regime, where unsupervised interaction with the environment is used to learn the dynamical distances, while a small amount of preference supervision is used to determine the task goal, without any manually engineered reward function or goal examples.
Meta-Learning with Implicit Gradients
By drawing upon implicit differentiation, we develop the implicit MAML algorithm, which depends only on the solution to the inner level optimization and not the path taken by the inner loop optimizer.
Scaled Autonomy: Enabling Human Operators to Control Robot Fleets
We learn a model of the user's preferences from observations of the user's choices in easy settings with a few robots, and use it in challenging settings with more robots to automatically identify which robot the user would most likely choose to control, if they were able to evaluate the states of all robots at all times.
Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
Hierarchical reinforcement learning has demonstrated significant success at solving difficult reinforcement learning (RL) tasks.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Recurrent Independent Mechanisms
Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes.
Consistent Meta-Reinforcement Learning via Model Identification and Experience Relabeling
Reinforcement learning algorithms can acquire policies for complex tasks automatically, however the number of samples required to learn a diverse set of skills can be prohibitively large.
