Search Results for author:
Found 41 papers, 16 papers with code
BarcodeBERT: Transformers for Biodiversity Analysis
Understanding biodiversity is a global challenge, in which DNA barcodes - short snippets of DNA that cluster by species - play a pivotal role.
Multi3DRefer: Grounding Text Description to Multiple 3D Objects
We introduce the task of localizing a flexible number of objects in real-world 3D scenes using natural language descriptions.
A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset
In an effort to catalog insect biodiversity, we propose a new large dataset of hand-labelled insect images, the BIOSCAN-Insect Dataset.
Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation
Surprisingly, we observe that agents trained on just 122 scenes from our dataset outperform agents trained on 10, 000 scenes from the ProcTHOR-10K dataset in terms of zero-shot generalization in real-world scanned environments.
Evaluating 3D Shape Analysis Methods for Robustness to Rotation Invariance
Our study, on a synthetic dataset of 3D scenes where objects instances occur in different orientations, reveals that deep learning-based rotation invariant methods are effective for relatively easy settings with easy-to-distinguish pairs.
MOPA: Modular Object Navigation with PointGoal Agents
We propose a simple but effective modular approach MOPA (Modular ObjectNav with PointGoal agents) to systematically investigate the inherent modularity of the object navigation task in Embodied AI.
Exploiting Proximity-Aware Tasks for Embodied Social Navigation
In this paper, we propose an end-to-end architecture that exploits Proximity-Aware Tasks (referred as to Risk and Proximity Compass) to inject into a reinforcement learning navigation policy the ability to infer common-sense social behaviors.
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding
Performing 3D dense captioning and visual grounding requires a common and shared understanding of the underlying multimodal relationships.
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Understanding Pure CLIP Guidance for Voxel Grid NeRF Models
We illustrate how different image-based augmentations prevent the adversarial generation problem, and how the generated results are impacted.
Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challenges
Human-object interactions with articulated objects are common in everyday life.
OPD: Single-view 3D Openable Part Detection
The input is a single image of an object, and as output we detect what parts of the object can open, and the motion parameters describing the articulation of each openable part.
TriCoLo: Trimodal Contrastive Loss for Fine-grained Text to Shape Retrieval
Recent work on contrastive losses for learning joint embeddings over multimodal data has been successful at downstream tasks such as retrieval and classification.
Roominoes: Generating Novel 3D Floor Plans From Existing 3D Rooms
We then discuss different strategies for solving the problem, and design two representative pipelines: one uses available 2D floor plans to guide selection and deformation of 3D rooms; the other learns to retrieve a set of compatible 3D rooms and combine them into novel layouts.
D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Grounding
Our D3Net unifies dense captioning and visual grounding in 3D in a self-critical manner.
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Prior work supervises the agent with actions based on the shortest path from the agent's location to the goal, but such goal-oriented supervision is often not in alignment with the instruction.
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
When compared to existing photorealistic 3D datasets such as Replica, MP3D, Gibson, and ScanNet, images rendered from HM3D have 20 - 85% higher visual fidelity w. r. t.
Mirror3D: Depth Refinement for Mirror Surfaces
Despite recent progress in depth sensing and 3D reconstruction, mirror surfaces are a significant source of errors.
Plan2Scene: Converting Floorplans to 3D Scenes
We address the task of converting a floorplan and a set of associated photos of a residence into a textured 3D mesh model, a task which we call Plan2Scene.
Ranked #1 on
Plan2Scene
on Rent3D++
MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
We propose the multiON task, which requires navigation to an episode-specific sequence of objects in a realistic environment.
Scan2Cap: Context-aware Dense Captioning in RGB-D Scans
We introduce the task of dense captioning in 3D scans from commodity RGB-D sensors.
Rearrangement: A Challenge for Embodied AI
In the rearrangement task, the goal is to bring a given physical environment into a specified state.
SAPIEN: A SimulAted Part-based Interactive ENvironment
To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language
We introduce the task of 3D object localization in RGB-D scans using natural language descriptions.
Mimic and Rephrase: Reflective Listening in Open-Ended Dialogue
Reflective listening{--}demonstrating that you have heard your conversational partner{--}is key to effective communication.
PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding
We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information.
Ranked #3 on
3D Instance Segmentation
on PartNet
Scan2CAD: Learning CAD Model Alignment in RGB-D Scans
For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry.
Ranked #1 on
3D Reconstruction
on Scan2CAD
Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View
We present Im2Pano3D, a convolutional neural network that generates a dense prediction of 3D structure and a probability distribution of semantic labels for a full 360 panoramic view of an indoor scene when given only a partial observation ( <=50%) in the form of an RGB-D image.
Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings
To this end, we first learn joint embeddings of freeform text descriptions and colored 3D shapes.
Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of View
We present Im2Pano3D, a convolutional neural network that generates a dense prediction of 3D structure and a probability distribution of semantic labels for a full 360 panoramic view of an indoor scene when given only a partial observation (<= 50%) in the form of an RGB-D image.
MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments
We present MINOS, a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments.
Learning Where to Look: Data-Driven Viewpoint Set Selection for 3D Scenes
We provide a search algorithm that generates a sampling of likely candidate views according to the example distribution, and a set selection algorithm that chooses a subset of the candidates that jointly cover the example distribution.
SceneSeer: 3D Scene Design with Natural Language
We present SceneSeer: an interactive text to 3D scene generation system that allows a user to design 3D scenes using natural language.
SceneSuggest: Context-driven 3D Scene Design
We present SceneSuggest: an interactive 3D scene design system providing context-driven suggestions for 3D model retrieval and placement.
Graphics Human-Computer Interaction
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets.
Ranked #11 on
Semantic Segmentation
on ScanNetV2
Semantic Scene Completion from a Single Depth Image
This paper focuses on semantic scene completion, a task for producing a complete 3D voxel representation of volumetric occupancy and semantic labels for a scene from a single-view depth map observation.
Ranked #2 on
3D Semantic Scene Completion
on KITTI-360
Evaluating the word-expert approach for Named-Entity Disambiguation
Named Entity Disambiguation (NED) is the task of linking a named-entity mention to an instance in a knowledge-base, typically Wikipedia.
ShapeNet: An Information-Rich 3D Model Repository
We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects.
SUTime: A library for recognizing and normalizing time expressions
We describe SUTIME, a temporal tagger for recognizing and normalizing temporal expressions in English text.
A Cross-Lingual Dictionary for English Wikipedia Concepts
We present a resource for automatically associating strings of text with English Wikipedia concepts.
