PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

PDF Abstract NeurIPS 2017 PDF NeurIPS 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Semantic Segmentation DALES PointNet++ mIoU 68.3 # 3
Overall Accuracy 95.7 # 6
Model size 3.0M # 5
Person Re-Identification DukeMTMC-reID PointNet++ (MSG) [qi2017pointnet++] Rank-1 60.23 # 80
mAP 39.36 # 85
3D Part Segmentation IntrA PointNet++ IoU (V) 93.42 # 5
DSC (V) 96.48 # 5
IoU (A) 76.38 # 4
DSC (A) 84.64 # 4
3D Semantic Segmentation KITTI-360 PointNet++ miou 35.66 # 3
mIoU Category 58.28 # 3
Model size 3.0M # 3
3D Point Cloud Classification ModelNet40 PointNet++ Overall Accuracy 90.7 # 87
Number of params 1.74M # 88