Search Results for author:
Found 23 papers, 13 papers with code
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
A long-term goal of AI research is to build intelligent agents that can communicate with humans in natural language, perceive the environment, and perform real-world tasks.
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges
However, current simulators for Embodied AI (EAI) challenges only provide simulated indoor scenes with a limited number of layouts.
TEACh: Task-driven Embodied Agents that Chat
Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes.
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion
Language-guided robots performing home and office tasks must navigate in and interact with the world.
Language Grounding with 3D Objects
We introduce several CLIP-based models for distinguishing objects and demonstrate that while recent advances in jointly modeling vision and language are useful for robotic language understanding, it is still the case that these image-based models are weaker at understanding the 3D nature of objects -- properties which play a key role in manipulation.
RMM: A Recursive Mental Model for Dialogue Navigation
In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers.
The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigation
In each trial, the pair first cooperates to localize the robot on a global map visible to the Commander, then the Driver follows Commander instructions to move the robot to a sequence of target objects.
RMM: A Recursive Mental Model for Dialog Navigation
In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers.
Experience Grounds Language
Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates.
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks.
Vision-and-Dialog Navigation
To train agents that search an environment for a goal location, we define the Navigation from Dialog History task.
Shifting the Baseline: Single Modality Performance on Visual Navigation \& QA
We demonstrate the surprising strength of unimodal baselines in multimodal domains, and make concrete recommendations for best practices in future research.
Improving Robot Success Detection using Static Object Data
We collect over 13 hours of egocentric manipulation data for training a model to reason about whether a robot successfully placed unseen objects in or on one another.
Interpreting Black Box Models via Hypothesis Testing
In science and medicine, model interpretations may be reported as discoveries of natural phenomena or used to guide patient treatments.
Prospection: Interpretable Plans From Language By Predicting the Future
High-level human instructions often correspond to behaviors with multiple implicit steps.
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Natural language understanding for robotics can require substantial domain- and platform-specific engineering.
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
We demonstrate the surprising strength of unimodal baselines in multimodal domains, and make concrete recommendations for best practices in future research.
Improving Black-box Speech Recognition using Semantic Parsing
Speech is a natural channel for human-computer interaction in robotics and consumer applications.
Automatic Speech Recognition
Natural Language Understanding
+2
Interpretable Low-Dimensional Regression via Data-Adaptive Smoothing
We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance.
Guiding Interaction Behaviors for Multi-modal Grounded Language Learning
Multi-modal grounded language learning connects language predicates to physical properties of objects in the world.
Integrated Learning of Dialog Strategies and Semantic Parsing
Natural language understanding and dialog management are two integral components of interactive dialog systems.
