Search Results for author:
Found 244 papers, 125 papers with code
Inferring Reward Functions from Demonstrators with Unknown Biases
Our goal is to infer reward functions from demonstrations.
Hierarchically Decoupled Morphological Transfer
Learning long-range behaviors on complex high-dimensional agents is a fundamental problem in robot learning.
CURL: Contrastive Unsupervised Representation Learning for Reinforcement Learning
CURL extracts high level features from raw pixels using a contrastive learning objective and performs off-policy control on top of the extracted features.
Planning to Explore via Latent Disagreement
To solve complex tasks, intelligent agents first need to explore their environments.
Responsive Safety in Reinforcement Learning
This intuition leads to our introduction of PID control for the Lagrange multiplier in constrained RL, which we cast as a dynamical system.
Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL
Meta-reinforcement learning (meta-RL) has proven to be a successful framework for leveraging experience from prior tasks to rapidly learn new related tasks, however, current meta-RL approaches struggle to learn in sparse reward environments.
Teachable Reinforcement Learning via Advice Distillation
Training automated agents to perform complex behaviors in interactive environments is challenging: reinforcement learning requires careful hand-engineering of reward functions, imitation learning requires specialized infrastructure and access to a human expert, and learning from intermediate forms of supervision (like binary preferences) is time-consuming and provides minimal information per human intervention.
Count-Based Temperature Scheduling for Maximum Entropy Reinforcement Learning
Maximum Entropy Reinforcement Learning (MaxEnt RL) algorithms such as Soft Q-Learning (SQL) and Soft Actor-Critic trade off reward and policy entropy, which has the potential to improve training stability and robustness.
Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning
We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size.
B-Pref: Benchmarking Preference-Based Reinforcement Learning
However, it is difficult to quantify the progress in preference-based RL due to the lack of a commonly adopted benchmark.
Mastering Atari Games with Limited Data
Recently, there has been significant progress in sample efficient image-based RL algorithms; however, consistent human-level performance on the Atari game benchmark remains an elusive goal.
URLB: Unsupervised Reinforcement Learning Benchmark
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm to solve a range of complex yet specific control tasks.
Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates
Under the belief that $\beta$ is closely related to the (state dependent) model uncertainty, Entropy Regularized Q-Learning (EQL) further introduces a principled scheduling of $\beta$ by maintaining a collection of the model parameters that characterizes model uncertainty.
Towards More Generalizable One-shot Visual Imitation Learning
We then study the multi-task setting, where multi-task training is followed by (i) one-shot imitation on variations within the training tasks, (ii) one-shot imitation on new tasks, and (iii) fine-tuning on new tasks.
APS: Active Pretraining with Successor Features
We introduce a new unsupervised pretraining objective for reinforcement learning.
Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback
A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations.
Playful Interactions for Representation Learning
In this work, we propose to use playful interactions in a self-supervised manner to learn visual representations for downstream tasks.
Hierarchical Few-Shot Imitation with Skill Transition Models
To this end, we present Few-shot Imitation with Skill Transition Models (FIST), an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks given a few downstream demonstrations.
The MineRL BASALT Competition on Learning from Human Feedback
Rather than training AI systems using a predefined reward function or using a labeled dataset with a predefined set of categories, we instead train the AI system using a learning signal derived from some form of human feedback, which can evolve over time as the understanding of the task changes, or as the capabilities of the AI system improve.
Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
Recent advance in deep offline reinforcement learning (RL) has made it possible to train strong robotic agents from offline datasets.
Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
Furthermore, in complex domains such as soccer, the space of possible scenarios is infinite, which makes it impossible for one research group to provide a comprehensive set of scenarios to train, test, and benchmark RL algorithms.
Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL
When combined together, they substantially improve the performance and generalization of offline RL policies.
Unsupervised Learning of Visual 3D Keypoints for Control
Prior works show that structured latent space such as visual keypoints often outperforms unstructured representations for robotic control.
Data-Efficient Exploration with Self Play for Atari
To alleviate the reliance on reward engineering it is important to develop RL algorithms capable of efficiently acquiring skills with no rewards extrinsic to the agent.
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.
JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data
Such a decomposition can dynamically control the reliability of information derived from the online and offline data and the use of pretrained neural networks permits scalability to large offline datasets.
Decision Transformer: Reinforcement Learning via Sequence Modeling
In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling.
Ranked #1 on
Atari Games
on Atari 2600 Pong
VideoGPT: Video Generation using VQ-VAE and Transformers
We present VideoGPT: a conceptually simple architecture for scaling likelihood based generative modeling to natural videos.
Ranked #2 on
Video Generation
on BAIR Robot Pushing
Auto-Tuned Sim-to-Real Transfer
Policies trained in simulation often fail when transferred to the real world due to the `reality gap' where the simulator is unable to accurately capture the dynamics and visual properties of the real world.
Learning What To Do by Simulating the Past
Since reward functions are hard to specify, recent work has focused on learning policies from human feedback.
GEM: Group Enhanced Model for Learning Dynamical Control Systems
In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning.
AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
Our system produces high-quality motions that are comparable to those achieved by state-of-the-art tracking-based techniques, while also being able to easily accommodate large datasets of unstructured motion clips.
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis
We present DietNeRF, a 3D neural scene representation estimated from a few images.
Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots
Developing robust walking controllers for bipedal robots is a challenging endeavor.
Mutual Information State Intrinsic Control
Reinforcement learning has been shown to be highly successful at many challenging tasks.
Pretrained Transformers as Universal Computation Engines
We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention and feedforward layers of the residual blocks.
Behavior From the Void: Unsupervised Active Pre-Training
We introduce a new unsupervised pre-training method for reinforcement learning called APT, which stands for Active Pre-Training.
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
Recent advances in off-policy deep reinforcement learning (RL) have led to impressive success in complex tasks from visual observations.
Ranked #31 on
Atari Games
on Atari 2600 Amidar
Task-Agnostic Morphology Evolution
Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form.
State Entropy Maximization with Random Encoders for Efficient Exploration
Recent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL).
MSA Transformer
Unsupervised protein language models trained across millions of diverse sequences learn structure and function of proteins.
Bottleneck Transformers for Visual Recognition
Finally, we present a simple adaptation of the BoTNet design for image classification, resulting in models that achieve a strong performance of 84. 7% top-1 accuracy on the ImageNet benchmark while being up to 1. 64x faster in compute time than the popular EfficientNet models on TPU-v3 hardware.
Ranked #18 on
Instance Segmentation
on COCO minival
Reinforcement Learning with Latent Flow
Temporal information is essential to learning effective policies with Reinforcement Learning (RL).
Discrete Predictive Representation for Long-horizon Planning
Discrete representations have been key in enabling robots to plan at more abstract levels and solve temporally-extended tasks more efficiently for decades.
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
As it turns out, fine-tuning offline RL agents is a non-trivial challenge, due to distribution shift – the agent encounters out-of-distribution samples during online interaction, which may cause bootstrapping error in Q-learning and instability during fine-tuning.
Unsupervised Active Pre-Training for Reinforcement Learning
On DMControl suite, APT beats all baselines in terms of asymptotic performance and data efficiency and dramatically improves performance on tasks that are extremely difficult for training from scratch.
Robust Imitation via Decision-Time Planning
The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal.
R-LAtte: Attention Module for Visual Control via Reinforcement Learning
Attention mechanisms are generic inductive biases that have played a critical role in improving the state-of-the-art in supervised learning, unsupervised pre-training and generative modeling for multiple domains including vision, language and speech.
Weighted Bellman Backups for Improved Signal-to-Noise in Q-Updates
Furthermore, since our weighted Bellman backups rely on maintaining an ensemble, we investigate how weighted Bellman backups interact with other benefits previously derived from ensembles: (a) Bootstrap; (b) UCB Exploration.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Compute- and Memory-Efficient Reinforcement Learning with Latent Experience Replay
In this paper, we present Latent Vector Experience Replay (LeVER), a simple modification of existing off-policy RL methods, to address these computational and memory requirements without sacrificing the performance of RL agents.
VideoGen: Generative Modeling of Videos using VQ-VAE and Transformers
We present VideoGen: a conceptually simple architecture for scaling likelihood based generative modeling to natural videos.
A Framework for Efficient Robotic Manipulation
Building on these advances, we present a Framework for Efficient Robotic Manipulation (FERM) that utilizes data augmentation and unsupervised learning to achieve extremely sample-efficient training of robotic manipulation policies with sparse rewards.
Parallel Training of Deep Networks with Local Updates
Deep learning models trained on large data sets have been widely successful in both vision and language domains.
Reset-Free Lifelong Learning with Skill-Space Planning
We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills.
Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning
Model-based reinforcement learning (RL) has shown great potential in various control tasks in terms of both sample-efficiency and final performance.
LaND: Learning to Navigate from Disengagements
However, we believe that these disengagements not only show where the system fails, which is useful for troubleshooting, but also provide a direct learning signal by which the robot can learn to navigate.
Decoupling Representation Learning from Reinforcement Learning
In an effort to overcome limitations of reward-driven feature learning in deep reinforcement learning (RL) from images, we propose decoupling representation learning from policy learning.
Visual Imitation Made Easy
We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector.
Robust Reinforcement Learning using Adversarial Populations
Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness, failing catastrophically when the underlying system dynamics are perturbed.
Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning
Despite the significant progress of deep reinforcement learning (RL) in solving sequential decision making problems, RL agents often overfit to training environments and struggle to adapt to new, unseen environments.
Hybrid Discriminative-Generative Training via Contrastive Learning
In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning.
Efficient Empowerment Estimation for Unsupervised Stabilization
We demonstrate our solution for sample-based unsupervised stabilization on different dynamical control systems and show the advantages of our method by comparing it to the existing VLB approaches.
Variable Skipping for Autoregressive Range Density Estimation
In this paper, we explore a technique, variable skipping, for accelerating range density estimation over deep autoregressive models.
Contrastive Code Representation Learning
Recent work learns contextual representations of source code by reconstructing tokens from their context.
Ranked #1 on
Method name prediction
on CodeSearchNet
SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning
Off-policy deep reinforcement learning (RL) has been successful in a range of challenging domains.
Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
Lagrangian methods are widely used algorithms for constrained optimization problems, but their learning dynamics exhibit oscillations and overshoot which, when applied to safe reinforcement learning, leads to constraint-violating behavior during agent training.
Self-Supervised Policy Adaptation during Deployment
A natural solution would be to keep training after deployment in the new environment, but this cannot be done if the new environment offers no reward signal.
AvE: Assistance via Empowerment
One difficulty in using artificial agents for human-assistive applications lies in the challenge of accurately assisting with a person's goal(s).
Locally Masked Convolution for Autoregressive Models
For tasks such as image completion, these models are unable to use much of the observed context.
Ranked #1 on
Image Generation
on MNIST
Denoising Diffusion Probabilistic Models
We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics.
Ranked #2 on
Image Generation
on LSUN Bedroom
Automatic Curriculum Learning through Value Disagreement
Our key insight is that if we can sample goals at the frontier of the set of goals that an agent is able to reach, it will provide a significantly stronger learning signal compared to randomly sampled goals.
Mutual Information Maximization for Robust Plannable Representations
The later, while they present low sample complexity, they learn latent spaces that need to reconstruct every single detail of the scene.
Model-Augmented Actor-Critic: Backpropagating through Paths
Current model-based reinforcement learning approaches use the model simply as a learned black-box simulator to augment the data for policy optimization or value function learning.
Planning to Explore via Self-Supervised World Models
Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge.
Plan2Vec: Unsupervised Representation Learning by Latent Plans
In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning.
Reinforcement Learning with Augmented Data
To this end, we present Reinforcement Learning with Augmented Data (RAD), a simple plug-and-play module that can enhance most RL algorithms.
CURL: Contrastive Unsupervised Representations for Reinforcement Learning
On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features.
Ranked #1 on
Continuous Control
on Finger, spin (DMControl500k)
Sparse Graphical Memory for Robust Planning
To operate effectively in the real world, agents should be able to act from high-dimensional raw sensory input such as images and achieve diverse goals across long time-horizons.
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
Using visual model-based learning for deformable object manipulation is challenging due to difficulties in learning plannable visual representations along with complex dynamic models.
Hierarchically Decoupled Imitation for Morphological Transfer
Learning long-range behaviors on complex high-dimensional agents is a fundamental problem in robot learning.
Hallucinative Topological Memory for Zero-Shot Visual Planning
In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline, e. g., images obtained from self-supervised robot interaction.
Generalized Hindsight for Reinforcement Learning
Compared to standard relabeling techniques, Generalized Hindsight provides a substantially more efficient reuse of samples, which we empirically demonstrate on a suite of multi-task navigation and manipulation tasks.
GACEM: Generalized Autoregressive Cross Entropy Method for Multi-Modal Black Box Constraint Satisfaction
In this work we present a new method of black-box optimization and constraint satisfaction.
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System
Mobile robot navigation is typically regarded as a geometric problem, in which the robot's objective is to perceive the geometry of the environment in order to plan collision-free paths towards a desired goal.
Mutual Information-based State-Control for Intrinsically Motivated Reinforcement Learning
In reinforcement learning, an agent learns to reach a set of goals by means of an external reward signal.
Preventing Imitation Learning with Adversarial Policy Ensembles
To our knowledge, this is the first work regarding the protection of policies in Reinforcement Learning.
Hierarchical Variational Imitation Learning of Control Programs
Autonomous agents can learn by imitating teacher demonstrations of the intended behavior.
Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge.
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
In this paper, we study how these challenges can be alleviated with an automated robotic learning framework, in which multi-stage tasks are defined simply by providing videos of a human demonstrator and then learned autonomously by the robot from raw image observations.
Learning Efficient Representation for Intrinsic Motivation
The core idea is to represent the relation between action sequences and future states using a stochastic dynamic model in latent space with a specific form.
Adaptive Online Planning for Continual Lifelong Learning
We study learning control in an online reset-free lifelong learning scenario, where mistakes can compound catastrophically into the future and the underlying dynamics of the environment may change.
Compositional Plan Vectors
We show that CPVs can be learned within a one-shot imitation learning framework without any additional supervision or information about task hierarchy, and enable a demonstration-conditioned policy to generalize to tasks that sequence twice as many skills as the tasks seen during training.
Natural Image Manipulation for Autoregressive Models Using Fisher Scores
Deep autoregressive models are one of the most powerful models that exist today which achieve state-of-the-art bits per dim.
Plan Arithmetic: Compositional Plan Vectors for Multi-Task Control
We show that CPVs can be learned within a one-shot imitation learning framework without any additional supervision or information about task hierarchy, and enable a demonstration-conditioned policy to generalize to tasks that sequence twice as many skills as the tasks seen during training.
Learning to Manipulate Deformable Objects without Demonstrations
Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points.
Asynchronous Methods for Model-Based Reinforcement Learning
In this work, we propose an asynchronous framework for model-based reinforcement learning methods that brings down the run time of these algorithms to be just the data collection time.
Geometry-Aware Neural Rendering
Understanding the 3-dimensional structure of the world is a core challenge in computer vision and robotics.
On the Utility of Learning about Humans for Human-AI Coordination
While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves.
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies.
PatchFormer: A neural architecture for self-supervised representation learning on images
In this paper, we propose a neural architecture for self-supervised representation learning on raw images called the PatchFormer which learns to model spatial dependencies across patches in a raw image.
Dynamical System Embedding for Efficient Intrinsically Motivated Artificial Agents
In this work, we develop a novel approach for the estimation of empowerment in unknown arbitrary dynamics from visual stimulus only, without sampling for the estimation of MIAS.
rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
rlpyt is designed as a high-throughput code base for small- to medium-scale research in deep RL.
DoorGym: A Scalable Door Opening Environment And Baseline Agent
We introduce DoorGym, an open-source door opening simulation framework designed to utilize domain randomization to train a stable policy.
Likelihood Contribution based Multi-scale Architecture for Generative Flows
An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture, which is based on iterative early factorization of a part of the total dimensions at regular intervals.
BagNet: Berkeley Analog Generator with Layout Optimizer Boosted with Deep Neural Networks
The discrepancy between post-layout and schematic simulation results continues to widen in analog design due in part to the domination of layout parasitics.
Benchmarking Model-Based Reinforcement Learning
Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL.
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
Deep reinforcement learning (RL) algorithms can use high-capacity deep networks to learn directly from image observations.
On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference
But in the era of deep learning, a natural suggestion researchers make is to avoid mathematical models of human behavior that are fraught with specific assumptions, and instead use a purely data-driven approach.
Evaluating Protein Transfer Learning with TAPE
Semi-supervised learning has emerged as an important paradigm in protein modeling due to the high cost of acquiring supervised protein labels, but the current literature is fragmented when it comes to datasets and standardized evaluation techniques.
Goal-conditioned Imitation Learning
Designing rewards for Reinforcement Learning (RL) is challenging because it needs to convey the desired task, be efficient to optimize, and be easy to compute.
Sub-policy Adaptation for Hierarchical Reinforcement Learning
Hierarchical reinforcement learning is a promising approach to tackle long-horizon decision-making problems with sparse rewards.
Learning latent state representation for speeding up exploration
In this work, we take a representation learning viewpoint on exploration, utilizing prior experience to learn effective latent representations, which can subsequently indicate which regions to explore.
MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies
In this work, we propose multiplicative compositional policies (MCP), a method for learning reusable motor skills that can be composed to produce a range of complex behaviors.
Compression with Flows via Local Bits-Back Coding
Likelihood-based generative models are the backbones of lossless compression due to the guaranteed existence of codes with lengths close to negative log likelihood.
Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables
The bits-back argument suggests that latent variable models can be turned into lossless compression schemes.
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
A key challenge in leveraging data augmentation for neural network training is choosing an effective augmentation policy from a large search space of candidate operations.
Ranked #7 on
Image Classification
on SVHN
Learning Robotic Manipulation through Visual Planning and Acting
We further demonstrate our approach on learning to imagine and execute in 3 environments, the final of which is deformable rope manipulation on a PR2 robot.
Deep Unsupervised Cardinality Estimation
To produce a truly usable estimator, we develop a Monte Carlo integration scheme on top of autoregressive models that can efficiently handle range queries with dozens of dimensions or more.
Learning to Reinforcement Learn by Imitation
Meta-reinforcement learning aims to learn fast reinforcement learning (RL) procedures that can be applied to new tasks or environments.
Guided Meta-Policy Search
Reinforcement learning (RL) algorithms have demonstrated promising results on complex tasks, yet often require impractical numbers of samples since they learn from scratch.
Towards Characterizing Divergence in Deep Q-Learning
Deep Q-Learning (DQL), a family of temporal difference algorithms for control, employs three techniques collectively known as the `deadly triad' in reinforcement learning: bootstrapping, off-policy learning, and function approximation.
Domain Randomization for Active Pose Estimation
In this work, we investigate how to improve the accuracy of domain randomization based pose estimation.
Preferences Implicit in the State of the World
We find that information from the initial state can be used to infer both side effects that should be avoided as well as preferences for how the environment should be organized.
Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight
Deep reinforcement learning provides a promising approach for vision-based control of real-world robots.
Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design
Flow-based generative models are powerful exact likelihood models with efficient sampling and inference.
Ranked #11 on
Image Generation
on ImageNet 32x32
Addressing Sample Complexity in Visual Tasks Using HER and Hallucinatory GANs
In this work, we show how visual trajectories can be hallucinated to appear successful by altering agent observations using a generative model trained on relatively few snapshots of the goal.
Soft Actor-Critic Algorithms and Applications
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
The Importance of Sampling inMeta-Reinforcement Learning
Results are presented on a new environment we call `Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning.
Guiding Policies with Language via Meta-Learning
However, a single instruction may be insufficient to fully communicate our intent or, even if it is, may be insufficient for an autonomous agent to actually understand how to perform the desired task.
An Algorithmic Perspective on Imitation Learning
This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning.
Modular Architecture for StarCraft II with Deep Reinforcement Learning
We present a novel modular architecture for StarCraft II AI.
One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks
We consider the problem of learning multi-stage vision-based tasks on a real robot from a single video of a human performing the task, while leveraging demonstration data of subtasks with other objects.
Establishing Appropriate Trust via Critical States
In order to effectively interact with or supervise a robot, humans need to have an accurate mental model of its capabilities and how it acts.
Robotics
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation
We show that a simulated robotic car and a real-world RC car can gather data and train fully autonomously without any human-provided labels beyond those needed to train the detectors, and then at test-time be able to accomplish a variety of different tasks.
ProMP: Proximal Meta-Policy Search
Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood.
SFV: Reinforcement Learning of Physical Skills from Videos
In this paper, we propose a method that enables physically simulated characters to learn skills from videos (SFV).
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients.
Model-Based Reinforcement Learning via Meta-Policy Optimization
Finally, we demonstrate that our approach is able to match the asymptotic performance of model-free methods while requiring significantly less experience.
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
Model-based reinforcement learning (RL) has proven to be a data efficient approach for learning control tasks but is difficult to utilize in domains with complex observations such as images.
Transfer Learning for Estimating Causal Effects using Neural Networks
We develop new algorithms for estimating heterogeneous treatment effects, combining recent developments in transfer learning for neural networks with insights from the causal inference literature.
Variational Option Discovery Algorithms
We explore methods for option discovery based on variational inference and make two algorithmic contributions.
Learning Plannable Representations with Causal InfoGAN
Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations.
Universal Planning Networks: Learning Generalizable Representations for Visuomotor Control
A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization.
Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
We show that we can learn continuous latent representations of trajectories, which are effective in solving temporally extended and multi-stage problems.
The Limits and Potentials of Deep Learning for Robotics
In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning.
Robotics
Latent Space Policies for Hierarchical Reinforcement Learning
In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acquires a range of diverse strategies via a maximum entropy reinforcement learning objective.
DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
We further explore a number of methods for integrating multiple clips into the learning process to develop multi-skilled agents capable of performing a rich repertoire of diverse skills.
Stochastic Adversarial Video Prediction
However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.
Ranked #4 on
Video Generation
on BAIR Robot Pushing
Universal Planning Networks
We find that the representations learned are not only effective for goal-directed visual imitation via gradient-based trajectory optimization, but can also provide a metric for specifying goals using images.
Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time.
Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines
To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP.
Learning Robotic Assembly from CAD
We propose to leverage this prior knowledge by guiding RL along a geometric motion plan, calculated using the CAD data.
Composable Deep Reinforcement Learning for Robotic Manipulation
Second, we show that policies learned with soft Q-learning can be composed to create new policies, and that the optimality of the resulting policy can be bounded in terms of the divergence between the composed policies.
Accelerated Methods for Deep Reinforcement Learning
Deep reinforcement learning (RL) has achieved many recent successes, yet experiment turn-around time remains a key bottleneck in research and in practice.
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
We consider the problem of exploration in meta reinforcement learning.
Model-Ensemble Trust-Region Policy Optimization
In this paper, we analyze the behavior of vanilla model-based reinforcement learning methods when deep neural networks are used to learn both the model and the policy, and show that the learned policy tends to exploit regions where insufficient data is available for the model to be learned, causing instability in training.
Meta-Reinforcement Learning of Structured Exploration Strategies
Exploration is a fundamental challenge in reinforcement learning (RL).
Evolved Policy Gradients
We propose a metalearning approach for learning gradient-based reinforcement learning (RL) algorithms.
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
Humans and animals are capable of learning a new behavior by observing others perform the skill just once.
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
A platform for Applied Reinforcement Learning (Applied RL)
Self-Supervised Learning of Object Motion Through Adversarial Video Prediction
In this paper, we study the problem of multi-step video prediction, where the goal is to predict a sequence of future frames conditioned on a short context.
PixelSNAIL: An Improved Autoregressive Generative Model
Autoregressive generative models consistently achieve the best results in density estimation tasks involving high dimensional data, such as images or audio.
A Berkeley View of Systems Challenges for AI
With the increasing commoditization of computer vision, speech recognition and machine translation systems and the widespread deployment of learning-based back-end technologies such as digital advertising and intelligent infrastructures, AI (Artificial Intelligence) has moved from research labs to production.
Safer Classification by Synthesis
We posit that a generative approach is the natural remedy for this problem, and propose a method for classification using generative models.
Inverse Reward Design
When designing the reward, we might think of some specific training scenarios, and make sure that the reward will lead to the right behavior in those scenarios.
Interpretable and Pedagogical Examples
Teachers intentionally pick the most informative examples to show their students.
Meta Learning Shared Hierarchies
We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives---policies that are executed for large numbers of timesteps.
Asymmetric Actor Critic for Image-Based Robot Learning
While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator.
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
By randomizing the dynamics of the simulator during training, we are able to develop policies that are capable of adapting to very different dynamics, including ones that differ significantly from the dynamics on which the policies were trained.
Robotics Systems and Control
Domain Randomization and Generative Models for Robotic Grasping
In this work, we explore a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis.
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Imitation learning is a powerful paradigm for robot skill acquisition.