Search Results for author: Devendra Singh Chaplot

Found 37 papers, 14 papers with code

GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation

no code implementations CVPR 2024 Mukul Khanna, Ram Ramrakhya, Gunjan Chhablani, Sriram Yenamandra, Theophile Gervet, Matthew Chang, Zsolt Kira, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi

The Embodied AI community has made significant strides in visual navigation tasks, exploring targets from 3D coordinates, objects, language descriptions, and images.

Navigate Visual Navigation

Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second

1 code implementation CVPR 2023 Vincent-Pierre Berges, Andrew Szot, Devendra Singh Chaplot, Aaron Gokaslan, Roozbeh Mottaghi, Dhruv Batra, Eric Undersander

Specifically, a Fetch robot (equipped with a mobile base, 7DoF arm, RGBD camera, egomotion, and onboard sensing) is spawned in a home environment and asked to rearrange objects - by navigating to an object, picking it up, navigating to a target location, and then placing the object at the target location.

Reinforcement Learning (RL)

AutoNeRF: Training Implicit Scene Representations with Autonomous Agents

1 code implementation21 Apr 2023 Pierre Marza, Laetitia Matignon, Olivier Simonin, Dhruv Batra, Christian Wolf, Devendra Singh Chaplot

Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment, and can be used for several downstream robotic tasks, and that modular trained exploration models outperform other classical and end-to-end baselines.

Novel View Synthesis

Navigating to Objects in the Real World

no code implementations2 Dec 2022 Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, Devendra Singh Chaplot

In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.

Navigate Visual Navigation

Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances

no code implementations29 Nov 2022 Jacob Krantz, Stefan Lee, Jitendra Malik, Dhruv Batra, Devendra Singh Chaplot

We consider the problem of embodied visual navigation given an image-goal (ImageNav) where an agent is initialized in an unfamiliar environment and tasked with navigating to a location 'described' by an image.

Visual Navigation

Multi-skill Mobile Manipulation for Object Rearrangement

no code implementations6 Sep 2022 Jiayuan Gu, Devendra Singh Chaplot, Hao Su, Jitendra Malik

To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks.

Object

PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning

no code implementations CVPR 2022 Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, Kristen Grauman

We propose Potential functions for ObjectGoal Navigation with Interaction-free learning (PONI), a modular approach that disentangles the skills of `where to look?'

Navigate

FILM: Following Instructions in Language with Modular Methods

1 code implementation ICLR 2022 So Yeon Min, Devendra Singh Chaplot, Pradeep Ravikumar, Yonatan Bisk, Ruslan Salakhutdinov

In contrast, we propose a modular method with structured representations that (1) builds a semantic map of the scene and (2) performs exploration with a semantic search policy, to achieve the natural language goal.

Imitation Learning Instruction Following

Building Intelligent Autonomous Navigation Agents

no code implementations25 Jun 2021 Devendra Singh Chaplot

In the first part of the thesis, we discuss our work on short-term navigation using end-to-end reinforcement learning to tackle challenges such as obstacle avoidance, semantic perception, language grounding, and reasoning.

Autonomous Navigation Decision Making +5

Planning with Submodular Objective Functions

no code implementations22 Oct 2020 Ruosong Wang, Hanrui Zhang, Devendra Singh Chaplot, Denis Garagić, Ruslan Salakhutdinov

We study planning with submodular objective functions, where instead of maximizing the cumulative reward, the goal is to maximize the objective value induced by a submodular function.

Object Goal Navigation using Goal-Oriented Semantic Exploration

2 code implementations NeurIPS 2020 Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov

We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category.

Object Robot Navigation

Semantic Curiosity for Active Visual Learning

no code implementations ECCV 2020 Devendra Singh Chaplot, Helen Jiang, Saurabh Gupta, Abhinav Gupta

Instead, we explore a self-supervised approach for training our exploration policy by introducing a notion of semantic curiosity.

Object object-detection +1

Neural Topological SLAM for Visual Navigation

no code implementations CVPR 2020 Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, Saurabh Gupta

This paper studies the problem of image-goal navigation which involves navigating to the location indicated by a goal image in a novel previously unseen environment.

Visual Navigation

Learning to Explore using Active Neural SLAM

2 code implementations ICLR 2020 Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, Ruslan Salakhutdinov

The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies).

PointGoal Navigation

Cross-Task Knowledge Transfer for Visually-Grounded Navigation

no code implementations ICLR 2019 Devendra Singh Chaplot, Lisa Lee, Ruslan Salakhutdinov, Devi Parikh, Dhruv Batra

Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for two different tasks: learning to follow navigational instructions and embodied question answering.

Disentanglement Embodied Question Answering +3

Embodied Multimodal Multitask Learning

no code implementations4 Feb 2019 Devendra Singh Chaplot, Lisa Lee, Ruslan Salakhutdinov, Devi Parikh, Dhruv Batra

In this paper, we propose a multitask model capable of jointly learning these multimodal tasks, and transferring knowledge of words and their grounding in visual objects across the tasks.

Disentanglement Embodied Question Answering +3

Learning Cognitive Models using Neural Networks

no code implementations21 Jun 2018 Devendra Singh Chaplot, Christopher MacLellan, Ruslan Salakhutdinov, Kenneth Koedinger

Secondly, for domains where a cognitive model is available, we show that representations learned through CogRL can be used to get accurate estimates of skill difficulty and learning rate parameters without using any student performance data.

Model Discovery

Gated Path Planning Networks

3 code implementations ICML 2018 Lisa Lee, Emilio Parisotto, Devendra Singh Chaplot, Eric Xing, Ruslan Salakhutdinov

Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture.

Active Neural Localization

1 code implementation ICLR 2018 Devendra Singh Chaplot, Emilio Parisotto, Ruslan Salakhutdinov

The results on the 2D environments show the effectiveness of the learned policy in an idealistic setting while results on the 3D environments demonstrate the model's capability of learning the policy and perceptual model jointly from raw-pixel based RGB observations.

Game of Doom

Knowledge-based Word Sense Disambiguation using Topic Models

no code implementations5 Jan 2018 Devendra Singh Chaplot, Ruslan Salakhutdinov

In this paper, we leverage the formalism of topic model to design a WSD system that scales linearly with the number of words in the context.

Sentence Topic Models +1

Gated-Attention Architectures for Task-Oriented Language Grounding

1 code implementation22 Jun 2017 Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, Ruslan Salakhutdinov

To perform tasks specified by natural language instructions, autonomous agents need to extract semantically meaningful representations of language and map it to visual elements and actions in the environment.

Imitation Learning

Playing FPS Games with Deep Reinforcement Learning

7 code implementations18 Sep 2016 Guillaume Lample, Devendra Singh Chaplot

Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions.

Game of Doom Q-Learning +2

Cannot find the paper you are looking for? You can Submit a new open access paper.