ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution

CVPR 2023  ·  Tuan Duc Ngo, Binh-Son Hua, Khoi Nguyen ·

Existing 3D instance segmentation methods are predominated by the bottom-up design -- manually fine-tuned algorithm to group points into clusters followed by a refinement network. However, by relying on the quality of the clusters, these methods generate susceptible results when (1) nearby objects with the same semantic class are packed together, or (2) large objects with loosely connected regions. To address these limitations, we introduce ISBNet, a novel cluster-free method that represents instances as kernels and decodes instance masks via dynamic convolution. To efficiently generate high-recall and discriminative kernels, we propose a simple strategy named Instance-aware Farthest Point Sampling to sample candidates and leverage the local aggregation layer inspired by PointNet++ to encode candidate features. Moreover, we show that predicting and leveraging the 3D axis-aligned bounding boxes in the dynamic convolution further boosts performance. Our method set new state-of-the-art results on ScanNetV2 (55.9), S3DIS (60.8), and STPLS3D (49.2) in terms of AP and retains fast inference time (237ms per scene on ScanNetV2). The source code and trained models are available at

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
3D Instance Segmentation S3DIS ISBNet mRec 77.1 # 1
mPrec 77.5 # 1
mCov 74.9 # 1
mWCov 76.8 # 1
AP@50 70.5 # 2
mAP 60.8 # 2
3D Instance Segmentation ScanNet200 ISBNet mAP 20.8 # 2
3D Instance Segmentation ScanNet(v2) ISBNet mAP 55.9 # 2
mAP @ 50 76.3 # 3
mAP@25 84.5 # 4
3D Instance Segmentation STPLS3D ISBNet AP50 64.0 # 2
AP 49.2 # 2