Search Results for author:
Found 61 papers, 31 papers with code
Hierarchically Decoupled Morphological Transfer
Learning long-range behaviors on complex high-dimensional agents is a fundamental problem in robot learning.
Behavior Generation with Latent Actions
Unlike language or image generation, decision making requires modeling actions - continuous-valued vectors that are multimodal in their distribution, potentially drawn from uncurated sources, where generation errors can compound in sequential prediction.
Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling
Reasoning from sequences of raw sensory data is a ubiquitous problem across fields ranging from medical devices to robotics.
OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics
The results demonstrate that OK-Robot achieves a 58. 5% success rate in open-ended pick-and-drop tasks, representing a new state-of-the-art in Open Vocabulary Mobile Manipulation (OVMM) with nearly 1. 8x the performance of prior work.
diff History for Neural Language Agents
On NetHack, an unsolved video game that requires long-horizon reasoning for decision-making, LMs tuned with diff history match state-of-the-art performance for neural agents while needing 1800x fewer training examples compared to prior work.
On Bringing Robots Home
We use the Stick to collect 13 hours of data in 22 homes of New York City, and train Home Pretrained Representations (HPR).
See to Touch: Learning Tactile Dexterity through Visual Incentives
First, we use a contrastive-based objective to learn visual representations.
Improving Long-Horizon Imitation Through Instruction Prediction
Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents.
Train Offline, Test Online: A Real Robot Learning Benchmark
Three challenges limit the progress of robot learning research: robots are expensive (few labs can participate), everyone uses different robots (findings do not generalize across labs), and we lack internet-scale robotics data.
Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play
In the first phase, we collect 2. 5 hours of play data, which is used to train self-supervised tactile encoders.
Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations
Given a weak base-policy trained by offline imitation of demonstrations, FISH computes rewards that correspond to the "match" between the robot's behavior and the demonstrations.
From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data
While large-scale sequence modeling from offline data has led to impressive performance gains in natural language and image generation, directly translating such ideas to robotics has been challenging.
Holo-Dex: Teaching Dexterity with Immersive Mixed Reality
A fundamental challenge in teaching robots is to provide an effective interface for human teachers to demonstrate useful skills to a robot.
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks, such as segmentation, instance identification, semantic search over space, and view localization.
That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulation
Learning to produce contact-rich, dynamic behaviors from raw sensory data has been a longstanding challenge in robotics.
Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment
In this work, we guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process.
Watch and Match: Supercharging Imitation with Regularized Optimal Transport
Our experiments on 20 visual control tasks across the DeepMind Control Suite, the OpenAI Robotics Suite, and the Meta-World Benchmark demonstrate an average of 7. 8X faster imitation to reach 90% of expert performance compared to prior state-of-the-art methods.
Behavior Transformers: Cloning $k$ modes with one stone
In this work, we present Behavior Transformer (BeT), a new technique to model unlabeled demonstration data with multiple modes.
Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation
Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature.
One After Another: Learning Incremental Skills for a Changing World
In this work, we propose a new framework for skill discovery, where skills are learned one after another in an incremental fashion.
Context is Everything: Implicit Identification for Dynamics Adaptation
Understanding environment dynamics is necessary for robots to act safely and optimally in the world.
Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
In this work, we propose Exploratory data for Offline RL (ExORL), a data-centric approach to offline RL.
The Surprising Effectiveness of Representation Learning for Visual Imitation
One reason such complexities arise is because standard visual imitation frameworks try to solve two coupled problems at once: learning a succinct but good representation from the diverse visual data, while simultaneously learning to associate the demonstrated actions with such representations.
URLB: Unsupervised Reinforcement Learning Benchmark
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm to solve a range of complex yet specific control tasks.
Improving Long-Horizon Imitation Through Language Prediction
Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents.
Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control.
Playful Interactions for Representation Learning
In this work, we propose to use playful interactions in a self-supervised manner to learn visual representations for downstream tasks.
GEM: Group Enhanced Model for Learning Dynamical Control Systems
In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning.
Simultaneous Navigation and Construction Benchmarking Environments
We need intelligent robots for mobile construction, the process of navigating in an environment and modifying its structure according to a geometric design.
Task-Agnostic Morphology Evolution
Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form.
Reinforcement Learning with Prototypical Representations
Unfortunately, in RL, representation learning is confounded with the exploratory experience of the agent -- learning a useful representation requires diverse data, while effective exploration is only possible with coherent representations.
Mobile Construction Benchmark
We need intelligent robots to perform mobile construction, the process of moving in an environment and modifying its geometry according to a design plan.
Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
We propose \textit{dynamics cycles} that align dynamic robot behavior across two domains using a cycle-consistency constraint.
Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
We present Contrastive Pre-training and Data Augmentation for Efficient Robotic Learning (CoDER), a method that utilizes data augmentation and unsupervised learning to achieve sample-efficient training of real-robot arm policies from sparse rewards.
Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
Vision-based robotics often separates the control loop into one module for perception and a separate module for control.
Visual Imitation Made Easy
We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector.
Self-Supervised Policy Adaptation during Deployment
A natural solution would be to keep training after deployment in the new environment, but this cannot be done if the new environment offers no reward signal.
Swoosh! Rattle! Thump! -- Actions that Sound
In this work, we perform the first large-scale study of the interactions between sound and robotic action.
Automatic Curriculum Learning through Value Disagreement
Our key insight is that if we can sample goals at the frontier of the set of goals that an agent is able to reach, it will provide a significantly stronger learning signal compared to randomly sampled goals.
Reinforcement Learning with Augmented Data
To this end, we present Reinforcement Learning with Augmented Data (RAD), a simple plug-and-play module that can enhance most RL algorithms.
State-Only Imitation Learning for Dexterous Manipulation
To tackle this setting, we train an inverse dynamics model and use it to predict actions for state-only demonstrations.
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
Using visual model-based learning for deformable object manipulation is challenging due to difficulties in learning plannable visual representations along with complex dynamic models.
Hierarchically Decoupled Imitation for Morphological Transfer
Learning long-range behaviors on complex high-dimensional agents is a fundamental problem in robot learning.
Generalized Hindsight for Reinforcement Learning
Compared to standard relabeling techniques, Generalized Hindsight provides a substantially more efficient reuse of samples, which we empirically demonstrate on a suite of multi-task navigation and manipulation tasks.
Discovering Motor Programs by Recomposing Demonstrations
In this paper, we present an approach to learn recomposable motor primitives across large-scale and diverse manipulation demonstrations.
Learning to Manipulate Deformable Objects without Demonstrations
Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points.
Swoosh! Rattle! Thump! - Actions that Sound
In this work, we perform the first large-scale study of the interactions between sound and robotic action.
Environment Probing Interaction Policies
A key challenge in reinforcement learning (RL) is environment generalization: a policy trained to solve a task in one environment often fails to solve the same task in a slightly different test environment.
PyRobot: An Open-source Robotics Framework for Research and Benchmarking
This paper introduces PyRobot, an open-source robotics framework for research and benchmarking.
Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributions
This paper proposes a sample-efficient yet simple approach to learning closed-loop policies for nonprehensile manipulation.
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation
In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc.
Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias
The models trained with our home dataset showed a marked improvement of 43. 7% over a baseline model trained with data collected in lab.
Asymmetric Actor Critic for Image-Based Robot Learning
While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator.
Predictive-State Decoders: Encoding the Future into Recurrent Networks
We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations.
CASSL: Curriculum Accelerated Self-Supervised Learning
Recent self-supervised learning approaches focus on using a few thousand data points to learn policies for high-level, low-dimensional action spaces.
Robust Adversarial Reinforcement Learning
Deep neural networks coupled with fast simulation and improved computation have led to recent successes in the field of reinforcement learning (RL).
Supervision via Competition: Robot Adversaries for Learning Tasks
Due to large number of experiences required for training, most of these approaches use a self-supervised paradigm: using sensors to measure success/failure.
Learning to Push by Grasping: Using multiple tasks for effective learning
The argument of the difficulty in scalability to multiple tasks is well founded, since training these tasks often require hundreds or thousands of examples.
The Curious Robot: Learning Visual Representations via Physical Interactions
We argue that biological agents use physical interactions with the world to learn visual representations unlike current vision systems which just use passive observations (images and videos downloaded from web).
Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours
Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping.
