3D Open-Vocabulary Instance Segmentation

8 papers with code • 4 benchmarks • 4 datasets

Open-vocabulary 3D instance segmentation is a computer vision task that involves identifying and delineating individual objects or instances within a three-dimensional (3D) scene without prior knowledge of a fixed set of object classes or categories. In other words, it extends traditional instance segmentation to a scenario where the number and types of objects present in the 3D environment are not predefined or limited to a specific vocabulary.

Most implemented papers

PointCLIP: Point Cloud Understanding by CLIP

zrrskywalker/pointclip CVPR 2022

On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.

PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning

yangyangyang127/pointclip_v2 ICCV 2023

In this paper, we first collaborate CLIP and GPT to be a unified 3D open-world learner, named as PointCLIP V2, which fully unleashes their potential for zero-shot 3D classification, segmentation, and detection.

OpenScene: 3D Scene Understanding with Open Vocabularies

pengsongyou/openscene CVPR 2023

Traditional 3D scene understanding approaches rely on labeled 3D datasets to train a model for a single task with supervision.

PLA: Language-Driven Open-Vocabulary 3D Scene Understanding

cvmi-lab/pla CVPR 2023

Open-vocabulary scene understanding aims to localize and recognize unseen categories beyond the annotated label space.

OpenMask3D: Open-Vocabulary 3D Instance Segmentation

OpenMask3D/openmask3d NeurIPS 2023

In this work, we address this limitation, and propose OpenMask3D, which is a zero-shot approach for open-vocabulary 3D instance segmentation.

OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation

Pointcept/OpenIns3D 1 Sep 2023

When integrated with powerful 2D open-world models such as ODISE and GroundingDINO, excellent results were observed on open-vocabulary instance segmentation.

OVIR-3D: Open-Vocabulary 3D Instance Retrieval Without Training on 3D Data

shiyoung77/ovir-3d 6 Nov 2023

This work presents OVIR-3D, a straightforward yet effective method for open-vocabulary 3D object instance retrieval without using any 3D data for training.

Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance

VinAIResearch/Open3DIS 17 Dec 2023

We introduce Open3DIS, a novel solution designed to tackle the problem of Open-Vocabulary Instance Segmentation within 3D scenes.