Search Results for author:
Found 37 papers, 12 papers with code
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks.
Evaluating Continual Learning on a Home Robot
Robots in home environments need to be able to learn new skills continuously as data becomes available, becoming ever more capable over time while using as little real-world data as possible.
HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation
In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations.
USA-Net: Unified Semantic and Affordance Representations for Robot Memory
In order for robots to follow open-ended instructions like "go open the brown cabinet over the sink", they require an understanding of both the scene geometry and the semantics of their environment.
Spatial-Language Attention Policies for Efficient Robot Learning
We see a 4x improvement over baseline in mobile manipulation setting.
Navigating to Objects Specified by Images
Our modular method solves sub-tasks of exploration, goal instance re-identification, goal localization, and local navigation.
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects
StructDiffusion even improves the success rate of assembling physically-valid structures out of unseen objects by on average 16% over an existing multi-modal transformer model trained on specific structures.
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks, such as segmentation, instance identification, semantic search over space, and view localization.
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
We analyze the performance of a set of baselines and show a correlation with a real-world evaluation.
Correcting Robot Plans with Natural Language Feedback
In this paper, we explore natural language as an expressive and flexible tool for robot correction.
Model Predictive Control for Fluid Human-to-Robot Handovers
However, how to responsively generate smooth motions to take an object from a human is still an open question.
Transporters with Visual Foresight for Solving Unseen Rearrangement Tasks
We propose a visual foresight model for pick-and-place rearrangement manipulation which is able to learn efficiently.
Pre-Trained Language Models for Interactive Decision-Making
Together, these results suggest that language modeling induces representations that are useful for modeling not just language, but also goals and plans; these representations can aid learning and generalization even outside of language processing.
IFOR: Iterative Flow Minimization for Robotic Object Rearrangement
Accurate object rearrangement from vision is a crucial problem for a wide variety of real-world robotics applications in unstructured environments.
Learning Perceptual Concepts by Bootstrapping from Human Queries
Second, we treat this low-dimensional concept as an automatic labeler to synthesize a large-scale high-dimensional data set with the simulator.
StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects
Geometric organization of objects into semantically meaningful arrangements pervades the built world.
SORNet: Spatial Object-Centric Representations for Sequential Manipulation
Sequential manipulation tasks require a robot to perceive the state of an environment and plan a sequence of actions leading to a desired goal state.
Predicting Stable Configurations for Semantic Placement of Novel Objects
We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.
Language Grounding with 3D Objects
We introduce several CLIP-based models for distinguishing objects and demonstrate that while recent advances in jointly modeling vision and language are useful for robotic language understanding, it is still the case that these image-based models are weaker at understanding the 3D nature of objects -- properties which play a key role in manipulation.
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents.
NeRP: Neural Rearrangement Planning for Unknown Objects
We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world.
Reactive Long Horizon Task Execution via Visual Skill and Precondition Models
Zero-shot execution of unseen robotic tasks is important to allowing robots to perform a wide variety of tasks in human environments, but collecting the amounts of data necessary to train end-to-end policies in the real-world is often infeasible.
Reactive Human-to-Robot Handovers of Arbitrary Objects
We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects, a user study with naive users (N=6) handing over a subset of 15 objects, and a systematic evaluation examining different ways of handing objects.
Human Grasp Classification for Reactive Human-to-Robot Handovers
In this paper, we propose an approach for human-to-robot handovers in which the robot meets the human halfway, by classifying the human's grasp of the object and quickly planning a trajectory accordingly to take the object from the human's hand according to their intent.
Transferable Task Execution from Pixels through Deep Planning Domain Learning
On the other hand, symbolic planning methods such as STRIPS have long been able to solve new problems given only a domain definition and a symbolic goal, but these approaches often struggle on the real world robotic tasks due to the challenges of grounding these symbols from sensor data in a partially-observable world.
6-DOF Grasping for Target-driven Object Manipulation in Clutter
Grasping in cluttered environments is a fundamental but challenging robotic skill.
Motion Reasoning for Goal-Based Imitation Learning
We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment.
Online Replanning in Belief Space for Partially Observable Task and Motion Problems
To solve multi-step manipulation tasks in the real world, an autonomous robot must take actions to observe its environment and react to unexpected observations.
Conditional Driving from Natural Language Instructions
Widespread adoption of self-driving cars will depend not only on their safety but largely on their ability to interact with human users.
"Good Robot!": Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer
We are able to create real stacks in 100% of trials with 61% efficiency and real rows in 100% of trials with 59% efficiency by directly loading the simulation-trained model on the real robot with no additional real-world fine-tuning.
Prospection: Interpretable Plans From Language By Predicting the Future
High-level human instructions often correspond to behaviors with multiple implicit steps.
The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints
We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances.
Visual Robot Task Planning
In this work, we propose a neural network architecture and associated planning algorithm that (1) learns a representation of the world useful for generating prospective futures after the application of high-level actions, (2) uses this generative model to simulate the result of sequences of high-level actions in a variety of environments, and (3) uses this same representation to evaluate these actions and perform tree search to find a sequence of high-level actions in a new environment.
Occupancy Map Prediction Using Generative and Fully Convolutional Networks for Vehicle Navigation
In these situations, the robot's ability to reason about its future motion is often severely limited by sensor field of view (FOV).
Learning to Imagine Manipulation Goals for Robot Task Planning
Ideally, we would combine the ability of machine learning to leverage big data for learning the semantics of a task, while using techniques from task planning to reliably generalize to new environment.
Temporal and Physical Reasoning for Perception-Based Robotic Manipulation
Accurate knowledge of object poses is crucial to successful robotic manipulation tasks, and yet most current approaches only work in laboratory settings.
Robotics
Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments
This paper investigates the ability of neural networks to learn both LTL constraints and control policies in order to generate task plans in complex environments.
Robotics
