Search Results for author:
Found 29 papers, 10 papers with code
Navigating to Objects in the Real World
In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances
We consider the problem of embodied visual navigation given an image-goal (ImageNav) where an agent is initialized in an unfamiliar environment and tasked with navigating to a location 'described' by an image.
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Habitat-Matterport 3D Semantics Dataset
The scale, quality, and diversity of object annotations far exceed those of prior datasets.
Multi-skill Mobile Manipulation for Object Rearrangement
To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks.
PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
We propose Potential functions for ObjectGoal Navigation with Interaction-free learning (PONI), a modular approach that disentangles the skills of `where to look?'
Recognizing Scenes from Novel Viewpoints
Humans can perceive scenes in 3D from a handful of 2D views.
SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency
The observations gathered by this exploration policy are labelled using 3D consistency and used to improve the perception model.
Differentiable Spatial Planning using Transformers
We consider the problem of spatial path planning.
FILM: Following Instructions in Language with Modular Methods
In contrast, we propose a modular method with structured representations that (1) builds a semantic map of the scene and (2) performs exploration with a semantic search policy, to achieve the natural language goal.
Building Intelligent Autonomous Navigation Agents
In the first part of the thesis, we discuss our work on short-term navigation using end-to-end reinforcement learning to tackle challenges such as obstacle avoidance, semantic perception, language grounding, and reasoning.
Unsupervised Domain Adaptation for Visual Navigation
We further show that our method can be used to transfer the navigation policies learned in simulation to the real world.
Planning with Submodular Objective Functions
We study planning with submodular objective functions, where instead of maximizing the cumulative reward, the goal is to maximize the objective value induced by a submodular function.
Object Goal Navigation using Goal-Oriented Semantic Exploration
We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category.
Ranked #4 on
Robot Navigation
on Habitat 2020 Object Nav test-std
Semantic Curiosity for Active Visual Learning
Instead, we explore a self-supervised approach for training our exploration policy by introducing a notion of semantic curiosity.
Neural Topological SLAM for Visual Navigation
This paper studies the problem of image-goal navigation which involves navigating to the location indicated by a goal image in a novel previously unseen environment.
Learning to Explore using Active Neural SLAM
The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies).
Cross-Task Knowledge Transfer for Visually-Grounded Navigation
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instructions and embodied question answering.
Embodied Multimodal Multitask Learning
In this paper, we propose a multitask model capable of jointly learning these multimodal tasks, and transferring knowledge of words and their grounding in visual objects across the tasks.
On Evaluation of Embodied Navigation Agents
Skillful mobile operation in three-dimensional environments is a primary topic of study in Artificial Intelligence.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
Learning Cognitive Models using Neural Networks
Secondly, for domains where a cognitive model is available, we show that representations learned through CogRL can be used to get accurate estimates of skill difficulty and learning rate parameters without using any student performance data.
Gated Path Planning Networks
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture.
Global Pose Estimation with an Attention-based Recurrent Network
The ability for an agent to localize itself within an environment is crucial for many real-world applications.
Active Neural Localization
The results on the 2D environments show the effectiveness of the learned policy in an idealistic setting while results on the 3D environments demonstrate the model's capability of learning the policy and perceptual model jointly from raw-pixel based RGB observations.
Knowledge-based Word Sense Disambiguation using Topic Models
In this paper, we leverage the formalism of topic model to design a WSD system that scales linearly with the number of words in the context.
Ranked #2 on
Word Sense Disambiguation
on Knowledge-based:
Gated-Attention Architectures for Task-Oriented Language Grounding
To perform tasks specified by natural language instructions, autonomous agents need to extract semantically meaningful representations of language and map it to visual elements and actions in the environment.
Playing FPS Games with Deep Reinforcement Learning
Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions.
