Search Results for author:
Found 17 papers, 7 papers with code
From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons
We examine the capability of Multimodal Large Language Models (MLLMs) to tackle diverse domains that extend beyond the traditional language and vision tasks these models are typically trained on.
ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI
Intelligent embodied agents need to quickly adapt to new scenarios by integrating long histories of experience into decision-making.
Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge
In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of objects across diverse environments.
Reinforcement Learning via Auxiliary Task Distillation
We present Reinforcement Learning via Auxiliary Task Distillation (AuxDistill), a new method that enables reinforcement learning (RL) to perform long-horizon robot control problems by distilling behaviors from auxiliary RL tasks.
Grounding Multimodal Large Language Models in Actions
For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance.
Large Language Models as Generalizable Policies for Embodied Tasks
We show that large language models (LLMs) can be adapted to be generalizable policies for embodied visual tasks.
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
We present Habitat 3. 0: a simulation platform for studying collaborative human-robot tasks in home environments.
Skill Transformer: A Monolithic Policy for Mobile Manipulation
We present Skill Transformer, an approach for solving long-horizon robotic tasks by combining conditional sequence modeling and skill modularity.
Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second
Specifically, a Fetch robot (equipped with a mobile base, 7DoF arm, RGBD camera, egomotion, and onboard sensing) is spawned in a home environment and asked to rearrange objects - by navigating to an object, picking it up, navigating to a target location, and then placing the object at the target location.
Adaptive Coordination in Social Embodied Rearrangement
We present the task of "Social Rearrangement", consisting of cooperative everyday tasks like setting up the dinner table, tidying a house or unpacking groceries in a simulated multi-agent environment.
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
How well do reward functions learned with inverse reinforcement learning (IRL) generalize?
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Housekeep: Tidying Virtual Households using Commonsense Reasoning
Instead, the agent must learn from and is evaluated against human preferences of which objects belong where in a tidy house.
Generalizable Imitation Learning from Observation via Inferring Goal Proximity
Task progress is intuitive and readily available task information that can guide an agent closer to the desired goal.
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
We introduce Habitat 2. 0 (H2. 0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios.
Goal-Driven Imitation Learning from Observation by Inferring Goal Proximity
Humans can effectively learn to estimate how close they are to completing a desired task simply by watching others fulfill the task.
Generalization to New Actions in Reinforcement Learning
A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices.
