PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation

Instance segmentation is an important task for scene understanding. Compared to the fully-developed 2D, 3D instance segmentation for point clouds have much room to improve. In this paper, we present PointGroup, a new end-to-end bottom-up architecture, specifically focused on better grouping the points by exploring the void space between objects. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. Further, we formulate the ScoreNet to evaluate the candidate instances, followed by the Non-Maximum Suppression (NMS) to remove duplicates. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best solutions in terms of mAP with IoU threshold 0.5.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Instance Segmentation S3DIS PointGroup mRec 69.2 # 9
mPrec 69.6 # 8
AP@50 64.0 # 10
3D Instance Segmentation ScanNet(v2) PointGroup mAP 40.7 # 18
mAP @ 50 63.6 # 17
3D Instance Segmentation STPLS3D PointGroup AP50 38.5 # 5
AP25 48.6 # 4
AP 23.3 # 5

Methods