Search Results for author:
Found 15 papers, 12 papers with code
Vision-Language Models Provide Promptable Representations for Reinforcement Learning
We find that our policies trained on embeddings extracted from general-purpose VLMs outperform equivalent policies trained on generic, non-promptable image embeddings.
Audio Visual Language Maps for Robot Navigation
While interacting in the world is a multi-sensory experience, many robots continue to predominantly rely on visual perception to map and navigate in their environments.
Visual Language Maps for Robot Navigation
Grounding language to the visual observations of a navigating agent can be performed using off-the-shelf visual-language models pretrained on Internet-scale data (e. g., image captions).
Grounding Language with Visual Affordances over Unstructured Data
Recent works have shown that Large Language Models (LLMs) can be applied to ground natural language to a wide variety of robot skills.
Ranked #1 on
Avg. sequence length
on CALVIN
Latent Plans for Task-Agnostic Offline Reinforcement Learning
Concretely, we combine a low-level policy that learns latent skills via imitation learning and a high-level policy learned from offline reinforcement learning for skill-chaining the latent behavior priors.
What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data
We have open-sourced our implementation to facilitate future research in learning to perform many complex manipulation skills in a row specified with natural language.
Affordance Learning from Play for Sample-Efficient Policy Learning
Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal.
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
We show that a baseline model based on multi-context imitation learning performs poorly on CALVIN, suggesting that there is significant room for developing innovative agents that learn to relate human language to their world models with this benchmark.
Composing Pick-and-Place Tasks By Grounding Language
Controlling robots to perform tasks via natural language is one of the most challenging topics in human-robot interaction.
Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces.
Learning Object Placements For Relational Instructions by Hallucinating Scene Representations
One particular requirement for such robots is that they are able to understand spatial relations and can place objects in accordance with the spatial relations expressed by their user.
Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video
Our method learns a general skill embedding independently from the task context by using an adversarial loss.
Self-supervised 3D Shape and Viewpoint Estimation from Single Images for Robotics
We present a convolutional neural network for joint 3D shape prediction and viewpoint estimation from a single input image.
3D Object Reconstruction From A Single Image
3D Reconstruction
+3
Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments
Object detection is an essential task for autonomous robots operating in dynamic and changing environments.
Metric Learning for Generalizing Spatial Relations to New Objects
Human-centered environments are rich with a wide variety of spatial relations between everyday objects.
