Search Results for author:
Found 212 papers, 95 papers with code
Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
In this work, we present a scalable reinforcement learning method for training multi-task policies from large offline datasets that can leverage both human demonstrations and autonomously collected data.
Robot Parkour Learning
Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly in complex environments.
BridgeData V2: A Dataset for Robot Learning at Scale
By publicly sharing BridgeData V2 and our pre-trained models, we aim to accelerate research in scalable robot learning methods.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
Waypoint-Based Imitation Learning for Robotic Manipulation
AWE can be combined with any BC algorithm, and we find that AWE can increase the success rate of state-of-the-art algorithms by up to 25% in simulation and by 4-28% on real-world bimanual manipulation tasks, reducing the decision making horizon by up to a factor of 10.
Contrastive Example-Based Control
In this paper, we propose a method for offline, example-based control that learns an implicit model of multi-step transitions, rather than a reward function.
Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrations
Eye-in-hand cameras have shown promise in enabling greater sample efficiency and generalization in vision-based robotic manipulation.
Polybot: Training One Policy Across Robots While Embracing Variability
Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets.
Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation
Towards an answer to this question, we study imitation learning policies in simulation and on a real robot language-conditioned manipulation task to quantify the difficulty of generalization to different (sets of) factors.
Supervised Pretraining Can Learn In-Context Reinforcement Learning
We find that the pretrained transformer can be used to solve a range of RL problems in-context, exhibiting both exploration online and conservatism offline, despite not being explicitly trained to do so.
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts
Effective machine learning models learn both robust features that directly determine the outcome of interest (e. g., an object with wheels is more likely to be a car), and shortcut features (e. g., an object on a road is more likely to be a car).
Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learning
Specifically, we design an office navigation environment, where the agent's goal is to find a particular office, and office locations differ in different buildings (i. e., tasks).
Conservative Prediction via Data-Driven Confidence Minimization
Taking inspiration from this result, we present data-driven confidence minimization (DCM), which minimizes confidence on an uncertainty dataset containing examples that the model is likely to misclassify at test time.
Train Offline, Test Online: A Real Robot Learning Benchmark
Three challenges limit the progress of robot learning research: robots are expensive (few labs can participate), everyone uses different robots (findings do not generalize across labs), and we lack internet-scale robotics data.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model.
Disentanglement via Latent Quantization
In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space.
Meta-Learning Online Adaptation of Language Models
Large language models encode surprisingly broad knowledge about the world into their parameters.
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
A trustworthy real-world prediction system should be well-calibrated; that is, its confidence in an answer is indicative of the likelihood that the answer is correct, enabling deferral to a more expensive expert in cases of low-confidence predictions.
Neural Functional Transformers
The recent success of neural networks as implicit representation of data has driven growing interest in neural functionals: models that can process other neural networks as input by operating directly over their weight spaces.
A Control-Centric Benchmark for Video Prediction
Video is a promising source of knowledge for embodied agents to learn models of the world's dynamics.
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback.
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors).
Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
In this paper, we study the fine-tuning problem in the context of conservative offline RL methods and we devise an approach for learning an effective initialization from offline data that also enables fast online fine-tuning capabilities.
Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning
In this work, we propose MEDAL++, a novel design for self-improving robotic systems: given a small set of expert demonstrations at the start, the robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
Open-World Object Manipulation using Pre-trained Vision-Language Models
This brings up a notably difficult challenge for robots: while robot learning approaches allow robots to learn many different behaviors from first-hand experience, it is impractical for robots to have first-hand experiences that span all of this semantic information.
Permutation Equivariant Neural Functionals
The key building blocks of this framework are NF-Layers (neural functional layers) that we constrain to be permutation equivariant through an appropriate parameter sharing scheme.
Language-Driven Representation Learning for Robotics
First, we demonstrate that existing representations yield inconsistent results across these tasks: masked autoencoding approaches pick up on low-level spatial features at the cost of high-level semantics, while contrastive learning approaches capture the opposite.
Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features
Transfer learning with a small amount of target data is an effective and common approach to adapting a pre-trained model to distribution shifts.
Leveraging Domain Relations for Domain Generalization
Distribution shift is a major challenge in machine learning, as models often perform poorly during the test stage if the test distribution differs from the training distribution.
Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts
Some robust training algorithms (e. g., Group DRO) specialize to group shifts and require group information on all training points.
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection.
A Survey of Meta-Reinforcement Learning
Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible.
NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis
Expert demonstrations are a rich source of supervision for training visual robotic manipulation policies, but imitation learning methods often require either a large number of demonstrations or expensive online expert supervision to learn reactive closed-loop behaviors.
RT-1: Robotics Transformer for Real-World Control at Scale
By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance.
Learning Options via Compression
Identifying statistical regularities in solutions to some tasks in multi-task reinforcement learning can accelerate the learning of new tasks.
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models
A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems.
Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time
Temporal shifts -- distribution shifts arising from the passage of time -- often occur gradually and have the additional structure of timestamp metadata.
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference
While large pre-trained language models are powerful, their predictions often lack logical consistency across test inputs.
Giving Feedback on Interactive Student Programs with Meta-Exploration
However, teaching and giving feedback on such software is time-consuming -- standard approaches require instructors to manually grade student-implemented interactive programs.
Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations
There is an inescapable long-tailed class-imbalance issue in many real-world classification problems.
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
A common approach to transfer learning under distribution shift is to fine-tune the last few layers of a pre-trained model, preserving learned features while also adapting to the new task.
When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning
A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world.
You Only Live Once: Single-Life Reinforcement Learning
We formalize this problem setting, which we call single-life reinforcement learning (SLRL), where an agent must complete a task within a single episode without interventions, utilizing its prior experience while contending with some form of novelty.
C-Mixup: Improving Generalization in Regression
In this paper, we propose a simple yet powerful algorithm, C-Mixup, to improve generalization on regression tasks.
Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials
To our knowledge, PTR is the first RL method that succeeds at learning new tasks in a new domain on a real WidowX robot with as few as 10 task demonstrations, by effectively leveraging an existing dataset of diverse multi-task robot data collected in a variety of toy kitchens.
Knowledge-Driven New Drug Recommendation
However, newly approved drugs do not have much historical prescription data and cannot leverage existing drug recommendation methods.
Learning to Reason With Relational Abstractions
Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.
Offline Reinforcement Learning at Multiple Frequencies
In this paper, we focus on one particular aspect of heterogeneity: learning from offline data collected at different control frequencies.
Memory-Based Model Editing at Scale
We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin.
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning
In a set of simulated tasks, we find that our system benefits from using audio, and that by using online interventions we are able to improve the success rate of offline imitation learning by ~20%.
A State-Distribution Matching Approach to Non-Episodic Reinforcement Learning
Prior works have considered an alternating approach where a forward policy learns to solve the task and the backward policy learns to reset the environment, but what initial state distribution should the backward policy reset the agent to?
Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models
We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.
R3M: A Universal Visual Representation for Robot Manipulation
We study how visual representations pre-trained on diverse human video data can enable data-efficient learning of downstream robotic manipulation tasks.
Do Deep Networks Transfer Invariances Across Classes?
Based on this analysis, we show how a generative approach for learning the nuisance transformations can help transfer invariances across classes and improve performance on a set of imbalanced image classification benchmarks.
Ranked #17 on
Long-tail Learning
on CIFAR-10-LT (ρ=100)
Latent-Variable Advantage-Weighted Policy Optimization for Offline RL
Offline reinforcement learning methods hold the promise of learning policies from pre-collected datasets without the need to query the environment for new transitions.
Vision-Based Manipulators Need to Also See from Their Hands
We study how the choice of visual perspective affects learning and generalization in the context of physical manipulation from raw sensor observations.
Policy Architectures for Compositional Generalization in Control
Many tasks in control, robotics, and planning can be specified using desired goal configurations for various entities in the environment.
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations
As ERM models can be good spurious attribute predictors, CNC works by (1) using a trained ERM model's outputs to identify samples with the same class but dissimilar spurious features, and (2) training a robust model with contrastive learning to learn similar representations for same-class samples.
Robust Policy Learning over Multiple Uncertainty Sets
Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.
Diversify and Disambiguate: Learning From Underspecified Data
Many datasets are underspecified: there exist multiple equally viable solutions to a given task.
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning.
How to Leverage Unlabeled Data in Offline Reinforcement Learning
One natural solution is to learn a reward function from the labeled data and use it to label the unlabeled data.
Fully Online Meta-Learning Without Task Boundaries
In this paper, we study how meta-learning can be applied to tackle online problems of this nature, simultaneously adapting to changing tasks and input distributions and meta-training the model in order to adapt more quickly in the future.
Improving Out-of-Distribution Robustness via Selective Augmentation
Machine learning algorithms typically assume that training and test examples are drawn from the same distribution.
Autonomous Reinforcement Learning: Formalism and Benchmarking
In this paper, we aim to address this discrepancy by laying out a framework for Autonomous Reinforcement Learning (ARL): reinforcement learning where the agent not only learns through its own experience, but also contends with lack of human supervision to reset between trials.
Extending the WILDS Benchmark for Unsupervised Adaptation
Unlabeled data can be a powerful point of leverage for mitigating these distribution shifts, as it is frequently much more available than labeled data and can often be obtained from distributions beyond the source distribution as well.
CoMPS: Continual Meta Policy Search
Beyond simply transferring past experience to new tasks, our goal is to devise continual reinforcement learning algorithms that learn to learn, using their experience on previous tasks to learn new tasks more quickly.
Information is Power: Intrinsic Control via Information Capture
We study this question in dynamic partially-observed environments, and argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.
MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance
Safe exploration is critical for using reinforcement learning (RL) in risk-sensitive environments.
Noether Networks: Meta-Learning Useful Conserved Quantities
Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases.
Meta-learning with an Adaptive Task Scheduler
In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta-model to unseen tasks.
Fast Model Editing at Scale
To enable easy post-hoc editing at scale, we propose Model Editor Networks using Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model's behavior.
MEMO: Test Time Robustness via Adaptation and Augmentation
We study the problem of test time robustification, i. e., using the test input to improve model robustness.
Data Sharing without Rewards in Multi-Task Offline Reinforcement Learning
However, these benefits come at a cost -- for data to be shared between tasks, each transition must be annotated with reward labels corresponding to other tasks.
FitVid: High-Capacity Pixel-Level Video Prediction
Furthermore, such an agent can internally represent the complex dynamics of the real-world and therefore can acquire a representation useful for a variety of visual perception tasks.
Test Time Robustification of Deep Models via Adaptation and Augmentation
We study the problem of test time robustification, i. e., using the test input to improve model robustness.
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Robot learning holds the promise of learning policies that generalize broadly.
A Workflow for Offline Model-Free Robotic Reinforcement Learning
To this end, we devise a set of metrics and conditions that can be tracked over the course of offline training, and can inform the practitioner about how the algorithm and model architecture should be adjusted to improve final performance.
Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks.
Model-based Reinforcement Learning
reinforcement-learning
+1
Lifelong Robotic Reinforcement Learning by Retaining Experiences
Multi-task learning ideally allows robots to acquire a diverse repertoire of useful skills.
Conservative Data Sharing for Multi-Task Offline Reinforcement Learning
We argue that a natural use case of offline RL is in settings where we can pool large amounts of data collected in various scenarios for solving different tasks, and utilize all of this data to learn behaviors for all the tasks more effectively rather than training each one in isolation.
Efficiently Identifying Task Groupings for Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation
However, goal images also have a number of drawbacks: they are inconvenient for humans to provide, they can over-specify the desired behavior leading to a sparse reward signal, or under-specify task information in the case of non-goal reaching tasks.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Bayesian Embeddings for Few-Shot Open World Recognition
As autonomous decision-making agents move from narrow operating environments to unstructured worlds, learning systems must move from a closed-world formulation to an open-world and few-shot setting in which agents continuously learn new classes from small amounts of information.
Autonomous Reinforcement Learning via Subgoal Curricula
Reinforcement learning (RL) promises to enable autonomous acquisition of complex behaviors for diverse agents.
ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
High-quality computer science education is limited by the difficulty of providing instructor feedback to students at scale.
Differentiable Annealed Importance Sampling and the Perils of Gradient Noise
To this end, we propose Differentiable AIS (DAIS), a variant of AIS which ensures differentiability by abandoning the Metropolis-Hastings corrections.
Just Train Twice: Improving Group Robustness without Training Group Information
Standard training via empirical risk minimization (ERM) can produce models that achieve high accuracy on average but low accuracy on certain groups, especially in the presence of spurious correlations between the input and label.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Visual Adversarial Imitation Learning using Variational Models
We consider a setting where an agent is provided a fixed dataset of visual demonstrations illustrating how to perform a task, and must learn to solve the task using the provided demonstrations and unsupervised environment interactions.
FitVid: Overfitting in Pixel-Level Video Prediction
There is a growing body of evidence that underfitting on the training data is one of the primary causes for the low quality predictions.
Ranked #6 on
Video Generation
on BAIR Robot Pushing
Intrinsic Control of Variational Beliefs in Dynamic Partially-Observed Visual Environments
We study this question in dynamic partially-observed environments, and argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.
Meta-Learning with Fewer Tasks through Task Interpolation
Meta-learning enables algorithms to quickly learn a newly encountered task with just a few labeled examples by transferring previously learned knowledge.
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
In this paper, we study how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously, sharing exploration, experience, and representations across tasks.
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills
We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data.
Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos
We find that by leveraging diverse human datasets, this reward function (a) can generalize zero shot to unseen environments, (b) generalize zero shot to unseen tasks, and (c) can be combined with visual model predictive control to solve robotic manipulation tasks on a real WidowX200 robot in an unseen environment from a single human demo.
Discriminator Augmented Model-Based Reinforcement Learning
By planning through a learned dynamics model, model-based reinforcement learning (MBRL) offers the prospect of good performance with little environment interaction.
Model-based Reinforcement Learning
reinforcement-learning
+1
FEW-SHOTLEARNING WITH WEAK SUPERVISION
Few-shot meta-learning methods aim to learn the common structure shared across a set of tasks to facilitate learning new tasks with small amounts of data.
Variational Model-Based Imitation Learning in High-Dimensional Observation Spaces
We consider the problem setting of imitation learning where the agent is provided a fixed dataset of demonstrations.
Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction
Our key insight is that greedy and modular optimization of hierarchical autoencoders can simultaneously address both the memory constraints and the optimization challenges of large-scale video prediction.
Ranked #1 on
Video Prediction
on Cityscapes 128x128
Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms
Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.
COMBO: Conservative Offline Model-Based Policy Optimization
We overcome this limitation by developing a new model-based offline RL algorithm, COMBO, that regularizes the value function on out-of-support state-action tuples generated via rollouts under the learned model.
How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains.
Variable-Shot Adaptation for Incremental Meta-Learning
Few-shot meta-learning methods consider the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.
On Trade-offs of Image Prediction in Visual Model-Based Reinforcement Learning
In this paper, we study a number of design decisions for the predictive model in visual MBRL algorithms, focusing specifically on methods that use a predictive model for planning.
Model-based Reinforcement Learning
reinforcement-learning
+1
Information Transfer in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
Model-Based Visual Planning with Self-Supervised Functional Distances
In our experiments, we find that our method can successfully learn models that perform a variety of tasks at test-time, moving objects amid distractors with a simulated robotic arm and even learning to open and close a drawer using a real-world robot.
Offline Reinforcement Learning from Images with Latent Space Models
In this work, we build on recent advances in model-based algorithms for offline RL, and extend them to high-dimensional visual observation spaces.
Variable-Shot Adaptation for Online Meta-Learning
Few-shot meta-learning methods consider the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Distribution shifts -- where the training distribution differs from the test distribution -- can substantially degrade the accuracy of machine learning (ML) systems deployed in the wild.
Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning
In this paper, we study a number of design decisions for the predictive model in visual MBRL algorithms, focusing specifically on methods that use a predictive model for planning.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Continual Learning of Control Primitives : Skill Discovery via Reset-Games
First, in real world settings, when an agent attempts a tasks and fails, the environment must somehow "reset" so that the agent can attempt the task again.
Learning Latent Representations to Influence Multi-Agent Interaction
We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy.
Reinforcement Learning with Videos: Combining Offline Observations with Interaction
In this paper, we consider the question: can we perform reinforcement learning directly on experience collected by humans?
Continual Learning of Control Primitives: Skill Discovery via Reset-Games
Reinforcement learning has the potential to automate the acquisition of behavior in complex settings, but in order for it to be successfully deployed, a number of practical challenges must be addressed.
Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones
Safety remains a central obstacle preventing widespread use of RL in the real world: learning new tasks in uncertain environments requires extensive exploration, but safety requires limiting exploration.
Measuring and Harnessing Transference in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL
While reinforcement learning algorithms can learn effective policies for complex tasks, these policies are often brittle to even minor task variations, especially when variations are not explicitly provided during training.
Learning to be Safe: Deep RL with a Safety Critic
Safety is an essential component for deploying reinforcement learning (RL) algorithms in real-world scenarios, and is critical during the learning process itself.
MELD: Meta-Reinforcement Learning from Images via Latent State Models
Meta-reinforcement learning algorithms can enable autonomous agents, such as robots, to quickly acquire new behaviors by leveraging prior experience in a set of related training tasks.
Batch Exploration with Examples for Scalable Robotic Reinforcement Learning
Concretely, we propose an exploration technique, Batch Exploration with Examples (BEE), that explores relevant regions of the state-space, guided by a modest number of human provided images of important states.
Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift
A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.
Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings
Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous, imperiling the RL agent, other agents, and the environment.
Offline Meta-Reinforcement Learning with Advantage Weighting
That is, in offline meta-RL, we meta-train on fixed, pre-collected data from several tasks in order to adapt to a new task with a very small amount (less than 5 trajectories) of data from the new task.
Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices
Learning a new task often requires both exploring to gather task-relevant information and exploiting this information to solve the task.
Goal-Aware Prediction: Learning to Model What Matters
In this paper, we propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space, resulting in a learning objective that more closely matches the downstream task.
Adaptive Risk Minimization: Learning to Adapt to Domain Shift
A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.
Meta-Learning Symmetries by Reparameterization
We present a method for learning and encoding equivariances into networks by learning corresponding parameter sharing patterns from data.
Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors
In this work we propose a framework for visual prediction and planning that is able to overcome both of these limitations.
Deep Reinforcement Learning amidst Lifelong Non-Stationarity
As humans, our goals and our environment are persistently changing throughout our lifetime based on our experiences, actions, and internal and external drives.
Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
Our method is based on a simple insight: we recognize that dynamics models can be adapted efficiently and consistently with off-policy data, more easily than policies and value functions.
Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation
One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments, but most robot learning systems today are deployed as fixed policies which do not adapt after deployment.
Explore then Execute: Adapting without Rewards via Factorized Meta-Reinforcement Learning
In principle, meta-reinforcement learning approaches can exploit this shared structure, but in practice, they fail to adapt to new environments when adaptation requires targeted exploration (e. g., exploring the cabinets to find ingredients in a new kitchen).
MOPO: Model-based Offline Policy Optimization
We also characterize the trade-off between the gain and risk of leaving the support of the batch data.
Watch, Try, Learn: Meta-Learning from Demonstrations and Rewards
Imitation learning allows agents to learn complex behaviors from demonstrations.
Model Based Reinforcement Learning for Atari
We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.
Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments.
Weakly-Supervised Reinforcement Learning for Controllable Behavior
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks.
OmniTact: A Multi-Directional High Resolution Touch Sensor
We compare with a state-of-the-art tactile sensor that is only sensitive on one side, as well as a state-of-the-art multi-directional tactile sensor, and find that OmniTact's combination of high-resolution and multi-directional sensing is crucial for reliably inserting the electrical connector and allows for higher accuracy in the state estimation task.
Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world.
Scalable Multi-Task Imitation Learning with Autonomous Improvement
In this work, we target this challenge, aiming to build an imitation learning system that can continuously improve through autonomous data collection, while simultaneously avoiding the explicit use of reinforcement learning, to maintain the stability, simplicity, and scalability of supervised imitation.
Gradient Surgery for Multi-Task Learning
While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge.
Learning Predictive Models From Observation and Interaction
Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes.
Continuous Meta-Learning without Tasks
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
SMiRL: Surprise Minimizing Reinforcement Learning in Unstable Environments
Every living organism struggles against disruptive environmental forces to carve out and maintain an orderly niche.
Meta-Learning without Memorization
If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes.
Unsupervised Curricula for Visual Meta-Reinforcement Learning
In experiments on vision-based navigation and manipulation domains, we show that the algorithm allows for unsupervised meta-learning that transfers to downstream tasks specified by hand-crafted reward functions and serves as pre-training for more efficient supervised meta-learning of test task distributions.
Entity Abstraction in Visual Model-Based Reinforcement Learning
This paper tests the hypothesis that modeling a scene in terms of entities and their local interactions, as opposed to modeling the scene globally, provides a significant benefit in generalizing to physical tasks in a combinatorial space the learner has not encountered before.
RoboNet: Large-Scale Multi-Robot Learning
This leads to a frequent tension in robotic learning: how can we learn generalizable robotic controllers without having to collect impractically large amounts of data for each separate experiment?
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Therefore, if the aim of these methods is to enable faster acquisition of entirely new behaviors, we must evaluate them on task distributions that are sufficiently broad to enable generalization to new behaviors.
Ranked #1 on
Meta-Learning
on ML10
Consistent Meta-Reinforcement Learning via Model Identification and Experience Relabeling
Reinforcement learning algorithms can acquire policies for complex tasks automatically, however the number of samples required to learn a diverse set of skills can be prohibitively large.
Goal-Conditioned Video Prediction
Prior work on video generation largely focuses on prediction models that only observe frames from the beginning of the video.
Mint: Matrix-Interleaving for Multi-Task Learning
Deep learning enables training of large and flexible function approximators from scratch at the cost of large amounts of data.