COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation

4 Oct 2022  ·  Rong Li, Anh-Quan Cao, Raoul de Charette ·

Annotation of large-scale 3D data is notoriously cumbersome and costly. As an alternative, weakly-supervised learning alleviates such a need by reducing the annotation by several order of magnitudes. We propose COARSE3D, a novel architecture-agnostic contrastive learning strategy for 3D segmentation. Since contrastive learning requires rich and diverse examples as keys and anchors, we leverage a prototype memory bank capturing class-wise global dataset information efficiently into a small number of prototypes acting as keys. An entropy-driven sampling technique then allows us to select good pixels from predictions as anchors. Experiments on three projection-based backbones show we outperform baselines on three challenging real-world outdoor datasets, working with as low as 0.001% annotations.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Weakly supervised Semantic Segmentation nuScenes COARSE3D mIoU 58.7 # 1
Weakly supervised Semantic Segmentation SemanticKITTI COARSE3D mIoU 55.7 # 1
Weakly supervised Semantic Segmentation SemanticPOSS COARSE3D mIoU 43.0 # 1