Search Results for author:
Found 27 papers, 8 papers with code
An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels
This work does not introduce a new method.
Exploitation-Guided Exploration for Semantic Embodied Navigation
In the recent progress in embodied navigation and sim-to-robot transfer, modular policies have emerged as a de facto framework.
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
We present Habitat 3. 0: a simulation platform for studying collaborative human-robot tasks in home environments.
An Unbiased Look at Datasets for Visuo-Motor Pre-Training
Visual representation learning hold great promise for robotics, but is severely hampered by the scarcity and homogeneity of robotics datasets.
Adaptive Coordination in Social Embodied Rearrangement
We present the task of "Social Rearrangement", consisting of cooperative everyday tasks like setting up the dinner table, tidying a house or unpacking groceries in a simulated multi-agent environment.
Affordances from Human Videos as a Versatile Representation for Robotics
Utilizing internet videos of human behavior, we train a visual affordance model that estimates where and how in the scene a human is likely to interact.
Pretrained Language Models as Visual Planners for Human Assistance
Given a succinct natural language goal, e. g., "make a shelf", and a video of the user's progress so far, the aim of VPA is to devise a plan, i. e., a sequence of actions such as "sand shelf", "paint shelf", etc.
Last-Mile Embodied Visual Navigation
Realistic long-horizon tasks like image-goal navigation involve exploratory and exploitative phases.
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Learning State-Aware Visual Representations from Audible Interactions
However, learning representations from videos can be challenging.
Ranked #3 on
Object State Change Classification
on Ego4D
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Prior work supervises the agent with actions based on the shortest path from the agent's location to the goal, but such goal-oriented supervision is often not in alignment with the instruction.
Cooperative Exploration for Multi-Agent Deep Reinforcement Learning
To address this shortcoming, in this paper, we propose cooperative multi-agent exploration (CMAE): agents share a common goal while exploring.
GridToPix: Training Embodied Agents with Minimal Supervision
While deep reinforcement learning (RL) promises freedom from hand-labeled data, great successes, especially for Embodied AI, require significant work to create supervision via carefully shaped rewards.
Coordinated Multi-Agent Exploration Using Shared Goals
Exploration is critical for good results of deep reinforcement learning algorithms and has drawn much attention.
MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
We propose the multiON task, which requires navigation to an episode-specific sequence of objects in a realistic environment.
AllenAct: A Framework for Embodied AI Research
The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement learning and increased interest from the computer vision, NLP, and robotics communities.
Bridging the Imitation Gap by Adaptive Insubordination
However, we show that when the teaching agent makes decisions with access to privileged information that is unavailable to the student, this information is marginalized during imitation learning, resulting in an "imitation gap" and, potentially, poor results.
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks
Autonomous agents must learn to collaborate.
SoundSpaces: Audio-Visual Navigation in 3D Environments
Moving around in the world is naturally a multisensory experience, but today's embodied agents are deaf---restricted to solely their visual perception of the environment.
TAB-VCR: Tags and Attributes based VCR Baselines
Despite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets.
TAB-VCR: Tags and Attributes based Visual Commonsense Reasoning Baselines
Despite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets.
Two Body Problem: Collaborative Visual Task Completion
Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities.
Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering
In addition, for the first time on the visual dialog dataset, we assess the performance of a system asking questions, and demonstrate how visual dialog can be generated from discriminative question generation and question answering.
Ranked #7 on
Visual Dialog
on VisDial v0.9 val
Compact Environment-Invariant Codes for Robust Visual Place Recognition
The modified system learns (in a supervised setting) compact binary codes from image feature descriptors.
Creativity: Generating Diverse Questions using Variational Autoencoders
Generating diverse questions for given images is an important task for computational education, entertainment and AI assistants.
