ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning

no code implementations • 28 Sep 2023 • Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, Liam Paull

We demonstrate the utility of this representation through a number of downstream planning tasks that are specified through abstract (language) prompts and require complex reasoning over spatial and semantic concepts.

ConceptFusion: Open-set Multimodal 3D Mapping

1 code implementation • 14 Feb 2023 • Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Alaa Maalouf, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, Ayush Tewari, Joshua B. Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, Antonio Torralba

ConceptFusion leverages the open-set capabilities of today's foundation models pre-trained on internet-scale data to reason about concepts across modalities such as natural language, images, and audio.

Autonomous Driving Robot Navigation

Augmenting Imitation Experience via Equivariant Representations

no code implementations • 14 Oct 2021 • Dhruv Sharma, Alihusein Kuwajerwala, Florian Shkurti

The robustness of visual navigation policies trained through imitation often hinges on the augmentation of the training image-action pairs.

Data Augmentation Visual Navigation