Search Results for author: Chelsea Finn

Found 212 papers, 95 papers with code

Robot Parkour Learning

no code implementations11 Sep 2023 Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christopher Atkeson, Soeren Schwertfeger, Chelsea Finn, Hang Zhao

Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly in complex environments.

Waypoint-Based Imitation Learning for Robotic Manipulation

no code implementations26 Jul 2023 Lucy Xiaoyang Shi, Archit Sharma, Tony Z. Zhao, Chelsea Finn

AWE can be combined with any BC algorithm, and we find that AWE can increase the success rate of state-of-the-art algorithms by up to 25% in simulation and by 4-28% on real-world bimanual manipulation tasks, reducing the decision making horizon by up to a factor of 10.

Imitation Learning

Contrastive Example-Based Control

1 code implementation24 Jul 2023 Kyle Hatch, Benjamin Eysenbach, Rafael Rafailov, Tianhe Yu, Ruslan Salakhutdinov, Sergey Levine, Chelsea Finn

In this paper, we propose a method for offline, example-based control that learns an implicit model of multi-step transitions, rather than a reward function.

Offline RL

Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrations

no code implementations12 Jul 2023 Moo Jin Kim, Jiajun Wu, Chelsea Finn

Eye-in-hand cameras have shown promise in enabling greater sample efficiency and generalization in vision-based robotic manipulation.

Domain Adaptation

Polybot: Training One Policy Across Robots While Embracing Variability

no code implementations7 Jul 2023 Jonathan Yang, Dorsa Sadigh, Chelsea Finn

Reusing large datasets is crucial to scale vision-based robotic manipulators to everyday scenarios due to the high cost of collecting robotic datasets.

Contrastive Learning

Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation

no code implementations7 Jul 2023 Annie Xie, Lisa Lee, Ted Xiao, Chelsea Finn

Towards an answer to this question, we study imitation learning policies in simulation and on a real robot language-conditioned manipulation task to quantify the difficulty of generalization to different (sets of) factors.

Imitation Learning

Supervised Pretraining Can Learn In-Context Reinforcement Learning

no code implementations26 Jun 2023 Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill

We find that the pretrained transformer can be used to solve a range of RL problems in-context, exhibiting both exploration online and conservatism offline, despite not being explicitly trained to do so.

Decision Making reinforcement-learning +1

Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts

no code implementations19 Jun 2023 Annie S. Chen, Yoonho Lee, Amrith Setlur, Sergey Levine, Chelsea Finn

Effective machine learning models learn both robust features that directly determine the outcome of interest (e. g., an object with wheels is more likely to be a car), and shortcut features (e. g., an object on a road is more likely to be a car).

Model Selection

Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learning

no code implementations14 Jun 2023 Evan Zheran Liu, Sahaana Suri, Tong Mu, Allan Zhou, Chelsea Finn

Specifically, we design an office navigation environment, where the agent's goal is to find a particular office, and office locations differ in different buildings (i. e., tasks).

Meta Reinforcement Learning Navigate +2

Conservative Prediction via Data-Driven Confidence Minimization

1 code implementation8 Jun 2023 Caroline Choi, Fahim Tajwar, Yoonho Lee, Huaxiu Yao, Ananya Kumar, Chelsea Finn

Taking inspiration from this result, we present data-driven confidence minimization (DCM), which minimizes confidence on an uncertainty dataset containing examples that the model is likely to misclassify at test time.

Train Offline, Test Online: A Real Robot Learning Benchmark

1 code implementation1 Jun 2023 Gaoyue Zhou, Victoria Dean, Mohan Kumar Srirama, Aravind Rajeswaran, Jyothish Pari, Kyle Hatch, Aryan Jain, Tianhe Yu, Pieter Abbeel, Lerrel Pinto, Chelsea Finn, Abhinav Gupta

Three challenges limit the progress of robot learning research: robots are expensive (few labs can participate), everyone uses different robots (findings do not generalize across labs), and we lack internet-scale robotics data.

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

4 code implementations29 May 2023 Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, Chelsea Finn

However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model.

Language Modelling reinforcement-learning

Disentanglement via Latent Quantization

1 code implementation28 May 2023 Kyle Hsu, Will Dorrell, James C. R. Whittington, Jiajun Wu, Chelsea Finn

In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space.

Disentanglement Inductive Bias +1

Meta-Learning Online Adaptation of Language Models

no code implementations24 May 2023 Nathan Hu, Eric Mitchell, Christopher D. Manning, Chelsea Finn

Large language models encode surprisingly broad knowledge about the world into their parameters.

Language Modelling Meta-Learning

Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback

no code implementations24 May 2023 Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, Christopher D. Manning

A trustworthy real-world prediction system should be well-calibrated; that is, its confidence in an answer is indicative of the likelihood that the answer is correct, enabling deferral to a more expensive expert in cases of low-confidence predictions.

Unsupervised Pre-training

Neural Functional Transformers

1 code implementation22 May 2023 Allan Zhou, KaiEn Yang, Yiding Jiang, Kaylee Burns, Winnie Xu, Samuel Sokota, J. Zico Kolter, Chelsea Finn

The recent success of neural networks as implicit representation of data has driven growing interest in neural functionals: models that can process other neural networks as input by operating directly over their weight spaces.

A Control-Centric Benchmark for Video Prediction

1 code implementation26 Apr 2023 Stephen Tian, Chelsea Finn, Jiajun Wu

Video is a promising source of knowledge for embodied agents to learn models of the world's dynamics.

Video Prediction

Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

no code implementations23 Apr 2023 Tony Z. Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn

Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback.

Chunking Imitation Learning

Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets

no code implementations18 Apr 2023 Maximilian Du, Suraj Nair, Dorsa Sadigh, Chelsea Finn

Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors).

Few-Shot Imitation Learning Imitation Learning +2

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

4 code implementations9 Mar 2023 Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, Sergey Levine

In this paper, we study the fine-tuning problem in the context of conservative offline RL methods and we devise an approach for learning an effective initialization from offline data that also enables fast online fine-tuning capabilities.

Offline RL Q-Learning +1

Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning

no code implementations2 Mar 2023 Archit Sharma, Ahmed M. Ahmed, Rehaan Ahmad, Chelsea Finn

In this work, we propose MEDAL++, a novel design for self-improving robotic systems: given a small set of expert demonstrations at the start, the robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.

reinforcement-learning Reinforcement Learning (RL)

Open-World Object Manipulation using Pre-trained Vision-Language Models

no code implementations2 Mar 2023 Austin Stone, Ted Xiao, Yao Lu, Keerthana Gopalakrishnan, Kuang-Huei Lee, Quan Vuong, Paul Wohlhart, Brianna Zitkovich, Fei Xia, Chelsea Finn, Karol Hausman

This brings up a notably difficult challenge for robots: while robot learning approaches allow robots to learn many different behaviors from first-hand experience, it is impractical for robots to have first-hand experiences that span all of this semantic information.

Language Modelling

Permutation Equivariant Neural Functionals

1 code implementation27 Feb 2023 Allan Zhou, KaiEn Yang, Kaylee Burns, Adriano Cardace, Yiding Jiang, Samuel Sokota, J. Zico Kolter, Chelsea Finn

The key building blocks of this framework are NF-Layers (neural functional layers) that we constrain to be permutation equivariant through an appropriate parameter sharing scheme.

Inductive Bias

Language-Driven Representation Learning for Robotics

2 code implementations24 Feb 2023 Siddharth Karamcheti, Suraj Nair, Annie S. Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, Percy Liang

First, we demonstrate that existing representations yield inconsistent results across these tasks: masked autoencoding approaches pick up on low-level spatial features at the cost of high-level semantics, while contrastive learning approaches capture the opposite.

Contrastive Learning Imitation Learning +2

Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features

no code implementations10 Feb 2023 Annie S. Chen, Yoonho Lee, Amrith Setlur, Sergey Levine, Chelsea Finn

Transfer learning with a small amount of target data is an effective and common approach to adapting a pre-trained model to distribution shifts.

Domain Adaptation Transfer Learning

Leveraging Domain Relations for Domain Generalization

no code implementations6 Feb 2023 Huaxiu Yao, Xinyu Yang, Xinyi Pan, Shengchao Liu, Pang Wei Koh, Chelsea Finn

Distribution shift is a major challenge in machine learning, as models often perform poorly during the test stage if the test distribution differs from the training distribution.

Domain Generalization

Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts

1 code implementation6 Feb 2023 Amrith Setlur, Don Dennis, Benjamin Eysenbach, aditi raghunathan, Chelsea Finn, Virginia Smith, Sergey Levine

Some robust training algorithms (e. g., Group DRO) specialize to group shifts and require group information on all training points.

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

2 code implementations26 Jan 2023 Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn

In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection.

Language Modelling Text Detection

A Survey of Meta-Reinforcement Learning

no code implementations19 Jan 2023 Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, Shimon Whiteson

Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible.

Meta Reinforcement Learning reinforcement-learning +1

NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis

no code implementations CVPR 2023 Allan Zhou, Moo Jin Kim, Lirui Wang, Pete Florence, Chelsea Finn

Expert demonstrations are a rich source of supervision for training visual robotic manipulation policies, but imitation learning methods often require either a large number of demonstrations or expensive online expert supervision to learn reactive closed-loop behaviors.

Data Augmentation Imitation Learning +2

Learning Options via Compression

1 code implementation8 Dec 2022 Yiding Jiang, Evan Zheran Liu, Benjamin Eysenbach, Zico Kolter, Chelsea Finn

Identifying statistical regularities in solutions to some tasks in multi-task reinforcement learning can accelerate the learning of new tasks.

Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models

1 code implementation27 Nov 2022 Peter Henderson, Eric Mitchell, Christopher D. Manning, Dan Jurafsky, Chelsea Finn

A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems.

Blocking Meta-Learning

Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time

1 code implementation25 Nov 2022 Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, Chelsea Finn

Temporal shifts -- distribution shifts arising from the passage of time -- often occur gradually and have the additional structure of timestamp metadata.

Continual Learning Domain Generalization +3

Giving Feedback on Interactive Student Programs with Meta-Exploration

1 code implementation16 Nov 2022 Evan Zheran Liu, Moritz Stephan, Allen Nie, Chris Piech, Emma Brunskill, Chelsea Finn

However, teaching and giving feedback on such software is time-consuming -- standard approaches require instructors to manually grade student-implemented interactive programs.

Surgical Fine-Tuning Improves Adaptation to Distribution Shifts

no code implementations20 Oct 2022 Yoonho Lee, Annie S. Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, Chelsea Finn

A common approach to transfer learning under distribution shift is to fine-tune the last few layers of a pre-trained model, preserving learned features while also adapting to the new task.

Transfer Learning

When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning

1 code implementation19 Oct 2022 Annie Xie, Fahim Tajwar, Archit Sharma, Chelsea Finn

A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world.

Continuous Control reinforcement-learning +1

You Only Live Once: Single-Life Reinforcement Learning

no code implementations17 Oct 2022 Annie S. Chen, Archit Sharma, Sergey Levine, Chelsea Finn

We formalize this problem setting, which we call single-life reinforcement learning (SLRL), where an agent must complete a task within a single episode without interventions, utilizing its prior experience while contending with some form of novelty.

Continuous Control reinforcement-learning +1

C-Mixup: Improving Generalization in Regression

1 code implementation11 Oct 2022 Huaxiu Yao, Yiping Wang, Linjun Zhang, James Zou, Chelsea Finn

In this paper, we propose a simple yet powerful algorithm, C-Mixup, to improve generalization on regression tasks.


Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials

1 code implementation11 Oct 2022 Aviral Kumar, Anikait Singh, Frederik Ebert, Mitsuhiko Nakamoto, Yanlai Yang, Chelsea Finn, Sergey Levine

To our knowledge, PTR is the first RL method that succeeds at learning new tasks in a new domain on a real WidowX robot with as few as 10 task demonstrations, by effectively leveraging an existing dataset of diverse multi-task robot data collected in a variety of toy kitchens.

Offline RL Q-Learning +1

Knowledge-Driven New Drug Recommendation

no code implementations11 Oct 2022 Zhenbang Wu, Huaxiu Yao, Zhe Su, David M Liebovitz, Lucas M Glass, James Zou, Chelsea Finn, Jimeng Sun

However, newly approved drugs do not have much historical prescription data and cannot leverage existing drug recommendation methods.

Few-Shot Learning Multi-Label Classification

Learning to Reason With Relational Abstractions

no code implementations6 Oct 2022 Andrew J. Nam, Mengye Ren, Chelsea Finn, James L. McClelland

Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.

Mathematical Reasoning

Offline Reinforcement Learning at Multiple Frequencies

no code implementations26 Jul 2022 Kaylee Burns, Tianhe Yu, Chelsea Finn, Karol Hausman

In this paper, we focus on one particular aspect of heterogeneity: learning from offline data collected at different control frequencies.

Offline RL reinforcement-learning +1

Memory-Based Model Editing at Scale

no code implementations13 Jun 2022 Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, Chelsea Finn

We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin.

Dialogue Generation Fact Checking +4

Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning

no code implementations30 May 2022 Maximilian Du, Olivia Y. Lee, Suraj Nair, Chelsea Finn

In a set of simulated tasks, we find that our system benefits from using audio, and that by using online interventions we are able to improve the success rate of offline imitation learning by ~20%.

Imitation Learning

A State-Distribution Matching Approach to Non-Episodic Reinforcement Learning

1 code implementation11 May 2022 Archit Sharma, Rehaan Ahmad, Chelsea Finn

Prior works have considered an alternating approach where a forward policy learns to solve the task and the backward policy learns to reset the environment, but what initial state distribution should the backward policy reset the agent to?

Continuous Control reinforcement-learning +1

Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models

no code implementations18 Apr 2022 Ali Ghadirzadeh, Petra Poklukar, Karol Arndt, Chelsea Finn, Ville Kyrki, Danica Kragic, Mårten Björkman

We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models.

Decision Making reinforcement-learning +2

R3M: A Universal Visual Representation for Robot Manipulation

1 code implementation23 Mar 2022 Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, Abhinav Gupta

We study how visual representations pre-trained on diverse human video data can enable data-efficient learning of downstream robotic manipulation tasks.

Contrastive Learning Robot Manipulation

Do Deep Networks Transfer Invariances Across Classes?

1 code implementation ICLR 2022 Allan Zhou, Fahim Tajwar, Alexander Robey, Tom Knowles, George J. Pappas, Hamed Hassani, Chelsea Finn

Based on this analysis, we show how a generative approach for learning the nuisance transformations can help transfer invariances across classes and improve performance on a set of imbalanced image classification benchmarks.

Image Classification Long-tail Learning

Latent-Variable Advantage-Weighted Policy Optimization for Offline RL

no code implementations16 Mar 2022 Xi Chen, Ali Ghadirzadeh, Tianhe Yu, Yuan Gao, Jianhao Wang, Wenzhe Li, Bin Liang, Chelsea Finn, Chongjie Zhang

Offline reinforcement learning methods hold the promise of learning policies from pre-collected datasets without the need to query the environment for new transitions.

Continuous Control Offline RL +2

Vision-Based Manipulators Need to Also See from Their Hands

no code implementations ICLR 2022 Kyle Hsu, Moo Jin Kim, Rafael Rafailov, Jiajun Wu, Chelsea Finn

We study how the choice of visual perspective affects learning and generalization in the context of physical manipulation from raw sensor observations.

Out-of-Distribution Generalization

Policy Architectures for Compositional Generalization in Control

no code implementations10 Mar 2022 Allan Zhou, Vikash Kumar, Chelsea Finn, Aravind Rajeswaran

Many tasks in control, robotics, and planning can be specified using desired goal configurations for various entities in the environment.

Imitation Learning Robot Manipulation

Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations

1 code implementation3 Mar 2022 Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang, Chelsea Finn, Christopher Ré

As ERM models can be good spurious attribute predictors, CNC works by (1) using a trained ERM model's outputs to identify samples with the same class but dissimilar spurious features, and (2) training a robust model with contrastive learning to learn similar representations for same-class samples.

Contrastive Learning

Robust Policy Learning over Multiple Uncertainty Sets

no code implementations14 Feb 2022 Annie Xie, Shagun Sodhani, Chelsea Finn, Joelle Pineau, Amy Zhang

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.

Reinforcement Learning (RL)

Diversify and Disambiguate: Learning From Underspecified Data

1 code implementation7 Feb 2022 Yoonho Lee, Huaxiu Yao, Chelsea Finn

Many datasets are underspecified: there exist multiple equally viable solutions to a given task.

Image Classification

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

no code implementations4 Feb 2022 Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, Chelsea Finn

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning.

Imitation Learning

Fully Online Meta-Learning Without Task Boundaries

no code implementations1 Feb 2022 Jathushan Rajasegaran, Chelsea Finn, Sergey Levine

In this paper, we study how meta-learning can be applied to tackle online problems of this nature, simultaneously adapting to changing tasks and input distributions and meta-training the model in order to adapt more quickly in the future.


Improving Out-of-Distribution Robustness via Selective Augmentation

2 code implementations2 Jan 2022 Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, Chelsea Finn

Machine learning algorithms typically assume that training and test examples are drawn from the same distribution.

Autonomous Reinforcement Learning: Formalism and Benchmarking

2 code implementations ICLR 2022 Archit Sharma, Kelvin Xu, Nikhil Sardana, Abhishek Gupta, Karol Hausman, Sergey Levine, Chelsea Finn

In this paper, we aim to address this discrepancy by laying out a framework for Autonomous Reinforcement Learning (ARL): reinforcement learning where the agent not only learns through its own experience, but also contends with lack of human supervision to reset between trials.

Benchmarking reinforcement-learning +1

Extending the WILDS Benchmark for Unsupervised Adaptation

1 code implementation ICLR 2022 Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

Unlabeled data can be a powerful point of leverage for mitigating these distribution shifts, as it is frequently much more available than labeled data and can often be obtained from distributions beyond the source distribution as well.

CoMPS: Continual Meta Policy Search

no code implementations ICLR 2022 Glen Berseth, Zhiwei Zhang, Grace Zhang, Chelsea Finn, Sergey Levine

Beyond simply transferring past experience to new tasks, our goal is to devise continual reinforcement learning algorithms that learn to learn, using their experience on previous tasks to learn new tasks more quickly.

Continual Learning Continuous Control +5

Information is Power: Intrinsic Control via Information Capture

no code implementations NeurIPS 2021 Nicholas Rhinehart, Jenny Wang, Glen Berseth, John D. Co-Reyes, Danijar Hafner, Chelsea Finn, Sergey Levine

We study this question in dynamic partially-observed environments, and argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.

Noether Networks: Meta-Learning Useful Conserved Quantities

no code implementations NeurIPS 2021 Ferran Alet, Dylan Doblar, Allan Zhou, Joshua Tenenbaum, Kenji Kawaguchi, Chelsea Finn

Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases.

Meta-Learning Translation

Meta-learning with an Adaptive Task Scheduler

1 code implementation NeurIPS 2021 Huaxiu Yao, Yu Wang, Ying WEI, Peilin Zhao, Mehrdad Mahdavi, Defu Lian, Chelsea Finn

In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the meta-model to unseen tasks.

Drug Discovery Meta-Learning

Fast Model Editing at Scale

2 code implementations ICLR 2022 Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, Christopher D. Manning

To enable easy post-hoc editing at scale, we propose Model Editor Networks using Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model's behavior.

Language Modelling Model Editing

MEMO: Test Time Robustness via Adaptation and Augmentation

2 code implementations18 Oct 2021 Marvin Zhang, Sergey Levine, Chelsea Finn

We study the problem of test time robustification, i. e., using the test input to improve model robustness.

Data Sharing without Rewards in Multi-Task Offline Reinforcement Learning

no code implementations29 Sep 2021 Tianhe Yu, Aviral Kumar, Yevgen Chebotar, Chelsea Finn, Sergey Levine, Karol Hausman

However, these benefits come at a cost -- for data to be shared between tasks, each transition must be annotated with reward labels corresponding to other tasks.

Multi-Task Learning Offline RL +2

FitVid: High-Capacity Pixel-Level Video Prediction

no code implementations29 Sep 2021 Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, Dumitru Erhan

Furthermore, such an agent can internally represent the complex dynamics of the real-world and therefore can acquire a representation useful for a variety of visual perception tasks.

Image Augmentation Video Prediction +1

Test Time Robustification of Deep Models via Adaptation and Augmentation

no code implementations29 Sep 2021 Marvin Mengxin Zhang, Sergey Levine, Chelsea Finn

We study the problem of test time robustification, i. e., using the test input to improve model robustness.

A Workflow for Offline Model-Free Robotic Reinforcement Learning

1 code implementation22 Sep 2021 Aviral Kumar, Anikait Singh, Stephen Tian, Chelsea Finn, Sergey Levine

To this end, we devise a set of metrics and conditions that can be tracked over the course of offline training, and can inform the practitioner about how the algorithm and model architecture should be adjusted to improve final performance.

Offline RL reinforcement-learning +1

Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

no code implementations21 Sep 2021 Bohan Wu, Suraj Nair, Li Fei-Fei, Chelsea Finn

In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks.

Model-based Reinforcement Learning reinforcement-learning +1

Conservative Data Sharing for Multi-Task Offline Reinforcement Learning

no code implementations NeurIPS 2021 Tianhe Yu, Aviral Kumar, Yevgen Chebotar, Karol Hausman, Sergey Levine, Chelsea Finn

We argue that a natural use case of offline RL is in settings where we can pool large amounts of data collected in various scenarios for solving different tasks, and utilize all of this data to learn behaviors for all the tasks more effectively rather than training each one in isolation.

Offline RL reinforcement-learning +1

Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation

no code implementations2 Sep 2021 Suraj Nair, Eric Mitchell, Kevin Chen, Brian Ichter, Silvio Savarese, Chelsea Finn

However, goal images also have a number of drawbacks: they are inconvenient for humans to provide, they can over-specify the desired behavior leading to a sparse reward signal, or under-specify task information in the case of non-goal reaching tasks.

On the Opportunities and Risks of Foundation Models

3 code implementations16 Aug 2021 Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, aditi raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang

AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.

Transfer Learning

Bayesian Embeddings for Few-Shot Open World Recognition

no code implementations29 Jul 2021 John Willes, James Harrison, Ali Harakeh, Chelsea Finn, Marco Pavone, Steven Waslander

As autonomous decision-making agents move from narrow operating environments to unstructured worlds, learning systems must move from a closed-world formulation to an open-world and few-shot setting in which agents continuously learn new classes from small amounts of information.

Decision Making Few-Shot Learning

ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback

1 code implementation23 Jul 2021 Mike Wu, Noah Goodman, Chris Piech, Chelsea Finn

High-quality computer science education is limited by the difficulty of providing instructor feedback to students at scale.

Few-Shot Learning

Differentiable Annealed Importance Sampling and the Perils of Gradient Noise

no code implementations NeurIPS 2021 Guodong Zhang, Kyle Hsu, Jianing Li, Chelsea Finn, Roger Grosse

To this end, we propose Differentiable AIS (DAIS), a variant of AIS which ensures differentiability by abandoning the Metropolis-Hastings corrections.

Stochastic Optimization

Just Train Twice: Improving Group Robustness without Training Group Information

1 code implementation19 Jul 2021 Evan Zheran Liu, Behzad Haghgoo, Annie S. Chen, aditi raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, Chelsea Finn

Standard training via empirical risk minimization (ERM) can produce models that achieve high accuracy on average but low accuracy on certain groups, especially in the presence of spurious correlations between the input and label.

Image Classification Out-of-Distribution Generalization

Visual Adversarial Imitation Learning using Variational Models

no code implementations NeurIPS 2021 Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, Chelsea Finn

We consider a setting where an agent is provided a fixed dataset of visual demonstrations illustrating how to perform a task, and must learn to solve the task using the provided demonstrations and unsupervised environment interactions.

Imitation Learning Representation Learning

FitVid: Overfitting in Pixel-Level Video Prediction

1 code implementation24 Jun 2021 Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, Dumitru Erhan

There is a growing body of evidence that underfitting on the training data is one of the primary causes for the low quality predictions.

Image Augmentation Video Generation +1

Intrinsic Control of Variational Beliefs in Dynamic Partially-Observed Visual Environments

no code implementations ICML Workshop URL 2021 Nicholas Rhinehart, Jenny Wang, Glen Berseth, John D Co-Reyes, Danijar Hafner, Chelsea Finn, Sergey Levine

We study this question in dynamic partially-observed environments, and argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.

Meta-Learning with Fewer Tasks through Task Interpolation

1 code implementation ICLR 2022 Huaxiu Yao, Linjun Zhang, Chelsea Finn

Meta-learning enables algorithms to quickly learn a newly encountered task with just a few labeled examples by transferring previously learned knowledge.

Image Classification Medical Image Classification +3

MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

no code implementations16 Apr 2021 Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, Karol Hausman

In this paper, we study how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously, sharing exploration, experience, and representations across tasks.

reinforcement-learning Reinforcement Learning (RL)

Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills

no code implementations15 Apr 2021 Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, Sergey Levine

We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data.

Q-Learning reinforcement-learning +1

Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos

no code implementations31 Mar 2021 Annie S. Chen, Suraj Nair, Chelsea Finn

We find that by leveraging diverse human datasets, this reward function (a) can generalize zero shot to unseen environments, (b) generalize zero shot to unseen tasks, and (c) can be combined with visual model predictive control to solve robotic manipulation tasks on a real WidowX200 robot in an unseen environment from a single human demo.

Discriminator Augmented Model-Based Reinforcement Learning

no code implementations24 Mar 2021 Behzad Haghgoo, Allan Zhou, Archit Sharma, Chelsea Finn

By planning through a learned dynamics model, model-based reinforcement learning (MBRL) offers the prospect of good performance with little environment interaction.

Model-based Reinforcement Learning reinforcement-learning +1


no code implementations ICLR Workshop Learning_to_Learn 2021 Ali Ghadirzadeh, Petra Poklukar, Xi Chen, Huaxiu Yao, Hossein Azizpour, Mårten Björkman, Chelsea Finn, Danica Kragic

Few-shot meta-learning methods aim to learn the common structure shared across a set of tasks to facilitate learning new tasks with small amounts of data.

Meta-Learning Variational Inference

Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction

no code implementations CVPR 2021 Bohan Wu, Suraj Nair, Roberto Martin-Martin, Li Fei-Fei, Chelsea Finn

Our key insight is that greedy and modular optimization of hierarchical autoencoders can simultaneously address both the memory constraints and the optimization challenges of large-scale video prediction.

Video Prediction

Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms

no code implementations5 Mar 2021 Ali Ghadirzadeh, Xi Chen, Petra Poklukar, Chelsea Finn, Mårten Björkman, Danica Kragic

Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.


COMBO: Conservative Offline Model-Based Policy Optimization

4 code implementations NeurIPS 2021 Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, Chelsea Finn

We overcome this limitation by developing a new model-based offline RL algorithm, COMBO, that regularizes the value function on out-of-support state-action tuples generated via rollouts under the learned model.

Offline RL

How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned

no code implementations4 Feb 2021 Julian Ibarz, Jie Tan, Chelsea Finn, Mrinal Kalakrishnan, Peter Pastor, Sergey Levine

Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains.

reinforcement-learning Reinforcement Learning (RL)

Variable-Shot Adaptation for Incremental Meta-Learning

no code implementations1 Jan 2021 Tianhe Yu, Xinyang Geng, Chelsea Finn, Sergey Levine

Few-shot meta-learning methods consider the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.

Meta-Learning Zero-Shot Learning

On Trade-offs of Image Prediction in Visual Model-Based Reinforcement Learning

no code implementations1 Jan 2021 Mohammad Babaeizadeh, Mohammad Taghi Saffar, Danijar Hafner, Dumitru Erhan, Harini Kannan, Chelsea Finn, Sergey Levine

In this paper, we study a number of design decisions for the predictive model in visual MBRL algorithms, focusing specifically on methods that use a predictive model for planning.

Model-based Reinforcement Learning reinforcement-learning +1

Information Transfer in Multi-Task Learning

no code implementations1 Jan 2021 Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn

Multi-task learning can leverage information learned by one task to benefit the training of other tasks.

Multi-Task Learning

Model-Based Visual Planning with Self-Supervised Functional Distances

1 code implementation ICLR 2021 Stephen Tian, Suraj Nair, Frederik Ebert, Sudeep Dasari, Benjamin Eysenbach, Chelsea Finn, Sergey Levine

In our experiments, we find that our method can successfully learn models that perform a variety of tasks at test-time, moving objects amid distractors with a simulated robotic arm and even learning to open and close a drawer using a real-world robot.

reinforcement-learning Reinforcement Learning (RL)

Offline Reinforcement Learning from Images with Latent Space Models

1 code implementation21 Dec 2020 Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, Chelsea Finn

In this work, we build on recent advances in model-based algorithms for offline RL, and extend them to high-dimensional visual observation spaces.

Offline RL reinforcement-learning +1

Variable-Shot Adaptation for Online Meta-Learning

no code implementations14 Dec 2020 Tianhe Yu, Xinyang Geng, Chelsea Finn, Sergey Levine

Few-shot meta-learning methods consider the problem of learning new tasks from a small, fixed number of examples, by meta-learning across static data from a set of previous tasks.

Meta-Learning Zero-Shot Learning

Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning

1 code implementation8 Dec 2020 Mohammad Babaeizadeh, Mohammad Taghi Saffar, Danijar Hafner, Harini Kannan, Chelsea Finn, Sergey Levine, Dumitru Erhan

In this paper, we study a number of design decisions for the predictive model in visual MBRL algorithms, focusing specifically on methods that use a predictive model for planning.

Model-based Reinforcement Learning Reinforcement Learning (RL)

Continual Learning of Control Primitives : Skill Discovery via Reset-Games

no code implementations NeurIPS 2020 Kelvin Xu, Siddharth Verma, Chelsea Finn, Sergey Levine

First, in real world settings, when an agent attempts a tasks and fails, the environment must somehow "reset" so that the agent can attempt the task again.

Continual Learning

Learning Latent Representations to Influence Multi-Agent Interaction

no code implementations12 Nov 2020 Annie Xie, Dylan P. Losey, Ryan Tolsma, Chelsea Finn, Dorsa Sadigh

We propose a reinforcement learning-based framework for learning latent representations of an agent's policy, where the ego agent identifies the relationship between its behavior and the other agent's future strategy.

Continual Learning of Control Primitives: Skill Discovery via Reset-Games

1 code implementation10 Nov 2020 Kelvin Xu, Siddharth Verma, Chelsea Finn, Sergey Levine

Reinforcement learning has the potential to automate the acquisition of behavior in complex settings, but in order for it to be successfully deployed, a number of practical challenges must be addressed.

Continual Learning

Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones

2 code implementations29 Oct 2020 Brijen Thananjeyan, Ashwin Balakrishna, Suraj Nair, Michael Luo, Krishnan Srinivasan, Minho Hwang, Joseph E. Gonzalez, Julian Ibarz, Chelsea Finn, Ken Goldberg

Safety remains a central obstacle preventing widespread use of RL in the real world: learning new tasks in uncertain environments requires extensive exploration, but safety requires limiting exploration.

reinforcement-learning Reinforcement Learning (RL) +1

Measuring and Harnessing Transference in Multi-Task Learning

no code implementations29 Oct 2020 Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn

Multi-task learning can leverage information learned by one task to benefit the training of other tasks.

Multi-Task Learning

One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

no code implementations NeurIPS 2020 Saurabh Kumar, Aviral Kumar, Sergey Levine, Chelsea Finn

While reinforcement learning algorithms can learn effective policies for complex tasks, these policies are often brittle to even minor task variations, especially when variations are not explicitly provided during training.

Learning to be Safe: Deep RL with a Safety Critic

no code implementations27 Oct 2020 Krishnan Srinivasan, Benjamin Eysenbach, Sehoon Ha, Jie Tan, Chelsea Finn

Safety is an essential component for deploying reinforcement learning (RL) algorithms in real-world scenarios, and is critical during the learning process itself.

Reinforcement Learning (RL) Transfer Learning

MELD: Meta-Reinforcement Learning from Images via Latent State Models

1 code implementation26 Oct 2020 Tony Z. Zhao, Anusha Nagabandi, Kate Rakelly, Chelsea Finn, Sergey Levine

Meta-reinforcement learning algorithms can enable autonomous agents, such as robots, to quickly acquire new behaviors by leveraging prior experience in a set of related training tasks.

Meta-Learning Meta Reinforcement Learning +3

Batch Exploration with Examples for Scalable Robotic Reinforcement Learning

1 code implementation22 Oct 2020 Annie S. Chen, HyunJi Nam, Suraj Nair, Chelsea Finn

Concretely, we propose an exploration technique, Batch Exploration with Examples (BEE), that explores relevant regions of the state-space, guided by a modest number of human provided images of important states.

Offline RL reinforcement-learning +1

Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift

no code implementations28 Sep 2020 Marvin Mengxin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, Chelsea Finn

A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.

Image Classification Meta-Learning

Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings

1 code implementation ICML 2020 Jesse Zhang, Brian Cheung, Chelsea Finn, Sergey Levine, Dinesh Jayaraman

Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous, imperiling the RL agent, other agents, and the environment.

reinforcement-learning Reinforcement Learning (RL)

Offline Meta-Reinforcement Learning with Advantage Weighting

2 code implementations13 Aug 2020 Eric Mitchell, Rafael Rafailov, Xue Bin Peng, Sergey Levine, Chelsea Finn

That is, in offline meta-RL, we meta-train on fixed, pre-collected data from several tasks in order to adapt to a new task with a very small amount (less than 5 trajectories) of data from the new task.

Machine Translation Meta-Learning +5

Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices

2 code implementations6 Aug 2020 Evan Zheran Liu, aditi raghunathan, Percy Liang, Chelsea Finn

Learning a new task often requires both exploring to gather task-relevant information and exploiting this information to solve the task.

Meta Reinforcement Learning reinforcement-learning +2

Goal-Aware Prediction: Learning to Model What Matters

no code implementations ICML 2020 Suraj Nair, Silvio Savarese, Chelsea Finn

In this paper, we propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space, resulting in a learning objective that more closely matches the downstream task.

Adaptive Risk Minimization: Learning to Adapt to Domain Shift

3 code implementations NeurIPS 2021 Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, Chelsea Finn

A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.

BIG-bench Machine Learning Domain Generalization +2

Meta-Learning Symmetries by Reparameterization

2 code implementations ICLR 2021 Allan Zhou, Tom Knowles, Chelsea Finn

We present a method for learning and encoding equivariances into networks by learning corresponding parameter sharing patterns from data.


Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors

1 code implementation NeurIPS 2020 Karl Pertsch, Oleh Rybkin, Frederik Ebert, Chelsea Finn, Dinesh Jayaraman, Sergey Levine

In this work we propose a framework for visual prediction and planning that is able to overcome both of these limitations.

Deep Reinforcement Learning amidst Lifelong Non-Stationarity

no code implementations ICML Workshop LifelongML 2020 Annie Xie, James Harrison, Chelsea Finn

As humans, our goals and our environment are persistently changing throughout our lifetime based on our experiences, actions, and internal and external drives.

reinforcement-learning Reinforcement Learning (RL)

Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling

no code implementations12 Jun 2020 Russell Mendonca, Xinyang Geng, Chelsea Finn, Sergey Levine

Our method is based on a simple insight: we recognize that dynamics models can be adapted efficiently and consistently with off-policy data, more easily than policies and value functions.

Meta Reinforcement Learning reinforcement-learning +1

Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation

no code implementations ICML Workshop LifelongML 2020 Ryan Julian, Benjamin Swanson, Gaurav S. Sukhatme, Sergey Levine, Chelsea Finn, Karol Hausman

One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments, but most robot learning systems today are deployed as fixed policies which do not adapt after deployment.

Continual Learning Robotic Grasping

Explore then Execute: Adapting without Rewards via Factorized Meta-Reinforcement Learning

no code implementations ICML Workshop LifelongML 2020 Evan Zheran Liu, aditi raghunathan, Percy Liang, Chelsea Finn

In principle, meta-reinforcement learning approaches can exploit this shared structure, but in practice, they fail to adapt to new environments when adaptation requires targeted exploration (e. g., exploring the cabinets to find ingredients in a new kitchen).

Meta Reinforcement Learning reinforcement-learning +2

Model Based Reinforcement Learning for Atari

no code implementations ICLR 2020 Łukasz Kaiser, Mohammad Babaeizadeh, Piotr Miłos, Błażej Osiński, Roy H. Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, Afroz Mohiuddin, Ryan Sepassi, George Tucker, Henryk Michalewski

We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.

Atari Games Model-based Reinforcement Learning +3

Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning

no code implementations21 Apr 2020 Ryan Julian, Benjamin Swanson, Gaurav S. Sukhatme, Sergey Levine, Chelsea Finn, Karol Hausman

One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments.

Continual Learning reinforcement-learning +2

OmniTact: A Multi-Directional High Resolution Touch Sensor

1 code implementation16 Mar 2020 Akhil Padmanabha, Frederik Ebert, Stephen Tian, Roberto Calandra, Chelsea Finn, Sergey Levine

We compare with a state-of-the-art tactile sensor that is only sensitive on one side, as well as a state-of-the-art multi-directional tactile sensor, and find that OmniTact's combination of high-resolution and multi-directional sensing is crucial for reliably inserting the electrical connector and allows for higher accuracy in the state estimation task.

Vocal Bursts Intensity Prediction

Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning

no code implementations2 Mar 2020 Xingyou Song, Yuxiang Yang, Krzysztof Choromanski, Ken Caluwaerts, Wenbo Gao, Chelsea Finn, Jie Tan

Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world.


Scalable Multi-Task Imitation Learning with Autonomous Improvement

no code implementations25 Feb 2020 Avi Singh, Eric Jang, Alexander Irpan, Daniel Kappler, Murtaza Dalal, Sergey Levine, Mohi Khansari, Chelsea Finn

In this work, we target this challenge, aiming to build an imitation learning system that can continuously improve through autonomous data collection, while simultaneously avoiding the explicit use of reinforcement learning, to maintain the stability, simplicity, and scalability of supervised imitation.

Imitation Learning reinforcement-learning +1

Gradient Surgery for Multi-Task Learning

9 code implementations NeurIPS 2020 Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, Chelsea Finn

While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge.

Image Classification Multi-Task Learning +1

Learning Predictive Models From Observation and Interaction

no code implementations ECCV 2020 Karl Schmeckpeper, Annie Xie, Oleh Rybkin, Stephen Tian, Kostas Daniilidis, Sergey Levine, Chelsea Finn

Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes.

Continuous Meta-Learning without Tasks

2 code implementations NeurIPS 2020 James Harrison, Apoorva Sharma, Chelsea Finn, Marco Pavone

In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.

Image Classification Meta-Learning +2

Meta-Learning without Memorization

1 code implementation ICLR 2020 Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, Chelsea Finn

If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes.

Few-Shot Image Classification Memorization +1

Unsupervised Curricula for Visual Meta-Reinforcement Learning

no code implementations NeurIPS 2019 Allan Jabri, Kyle Hsu, Ben Eysenbach, Abhishek Gupta, Sergey Levine, Chelsea Finn

In experiments on vision-based navigation and manipulation domains, we show that the algorithm allows for unsupervised meta-learning that transfers to downstream tasks specified by hand-crafted reward functions and serves as pre-training for more efficient supervised meta-learning of test task distributions.

Clustering Meta-Learning +3

Entity Abstraction in Visual Model-Based Reinforcement Learning

1 code implementation28 Oct 2019 Rishi Veerapaneni, John D. Co-Reyes, Michael Chang, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua B. Tenenbaum, Sergey Levine

This paper tests the hypothesis that modeling a scene in terms of entities and their local interactions, as opposed to modeling the scene globally, provides a significant benefit in generalizing to physical tasks in a combinatorial space the learner has not encountered before.

Model-based Reinforcement Learning Object Discovery +4

RoboNet: Large-Scale Multi-Robot Learning

no code implementations24 Oct 2019 Sudeep Dasari, Frederik Ebert, Stephen Tian, Suraj Nair, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh, Sergey Levine, Chelsea Finn

This leads to a frequent tension in robotic learning: how can we learn generalizable robotic controllers without having to collect impractically large amounts of data for each separate experiment?

Video Prediction

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

8 code implementations24 Oct 2019 Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, Sergey Levine

Therefore, if the aim of these methods is to enable faster acquisition of entirely new behaviors, we must evaluate them on task distributions that are sufficiently broad to enable generalization to new behaviors.

Meta-Learning Meta Reinforcement Learning +3

Consistent Meta-Reinforcement Learning via Model Identification and Experience Relabeling

no code implementations25 Sep 2019 Russell Mendonca, Xinyang Geng, Chelsea Finn, Sergey Levine

Reinforcement learning algorithms can acquire policies for complex tasks automatically, however the number of samples required to learn a diverse set of skills can be prohibitively large.

Meta Reinforcement Learning reinforcement-learning +1

Goal-Conditioned Video Prediction

no code implementations25 Sep 2019 Oleh Rybkin, Karl Pertsch, Frederik Ebert, Dinesh Jayaraman, Chelsea Finn, Sergey Levine

Prior work on video generation largely focuses on prediction models that only observe frames from the beginning of the video.

Imitation Learning Video Generation +1

Mint: Matrix-Interleaving for Multi-Task Learning

no code implementations25 Sep 2019 Tianhe Yu, Saurabh Kumar, Eric Mitchell, Abhishek Gupta, Karol Hausman, Sergey Levine, Chelsea Finn

Deep learning enables training of large and flexible function approximators from scratch at the cost of large amounts of data.

Multi-Task Learning reinforcement-learning +1