Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering

12 Jan 2024  ·  Damien Robert, Hugo Raguet, Loic Landrieu ·

We introduce a highly efficient method for panoptic segmentation of large 3D point clouds by redefining this task as a scalable graph clustering problem. This approach can be trained using only local auxiliary tasks, thereby eliminating the resource-intensive instance-matching step during training. Moreover, our formulation can easily be adapted to the superpoint paradigm, further increasing its efficiency. This allows our model to process scenes with millions of points and thousands of objects in a single inference. Our method, called SuperCluster, achieves a new state-of-the-art panoptic segmentation performance for two indoor scanning datasets: $50.1$ PQ ($+7.8$) for S3DIS Area~5, and $58.7$ PQ ($+25.2$) for ScanNetV2. We also set the first state-of-the-art for two large-scale mobile mapping benchmarks: KITTI-360 and DALES. With only $209$k parameters, our model is over $30$ times smaller than the best-competing method and trains up to $15$ times faster. Our code and pretrained models are available at https://github.com/drprojects/superpoint_transformer.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Panoptic Segmentation DALES SuperCluster PQ 61.2 # 1
RQ 68.6 # 1
SQ 87.1 # 1
Params (M) 0.21 # 1
3D Semantic Segmentation DALES SuperCluster mIoU 77.3 # 3
Model size 210M # 8
Panoptic Segmentation KITTI-360 SuperCluster PQ 48.3 # 1
RQ 58.4 # 1
SQ 75.1 # 1
Params (M) 0.79 # 1
3D Semantic Segmentation KITTI-360 SuperCluster miou Val 62.1 # 2
Model size 790K # 1
Panoptic Segmentation S3DIS SuperCluster PQ 55.9 # 1
RQ 66.3 # 1
SQ 83.8 # 1
PQ (with stuff) 62.7 # 1
RQ (with stuff) 73.2 # 1
SQ (with stuff) 84.8 # 1
Params (M) 0.21 # 1
Semantic Segmentation S3DIS SuperCluster Mean IoU 75.3 # 10
Number of params 0.21M # 35
Params (M) 0.21 # 18
Semantic Segmentation S3DIS Area5 SuperCluster mIoU 68.1 # 30
Number of params 0.21 # 1
Panoptic Segmentation S3DIS Area5 SuperCluster PQ 50.1 # 1
RQ 60.1 # 1
SQ 76.6 # 1
PQ (with stuff) 58.4 # 1
RQ (with stuff) 68.4 # 1
SQ (with stuff) 77.8 # 1
Params (M) 0.21 # 1
Panoptic Segmentation ScanNet SuperCluster PQ 58.7 # 2
PQ_th 69.1 # 2
PQ_st 84.1 # 2
Panoptic Segmentation ScanNetV2 SuperCluster PQ 58.7 # 3
SQ 84.1 # 1
RQ 69.1 # 1
Params (M) 1 # 1

Methods


No methods listed for this paper. Add relevant methods here