PointCNN: Convolution On X-Transformed Points

We present a simple and general framework for feature learning from point cloud. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point cloud are irregular and unordered, thus a direct convolving of kernels against the features associated with the points will result in deserting the shape information while being variant to the orders. To address these problems, we propose to learn a X-transformation from the input points, which is used for simultaneously weighting the input features associated with the points and permuting them into latent potentially canonical order. Then element-wise product and sum operations of typical convolution operator are applied on the X-transformed features. The proposed method is a generalization of typical CNNs into learning features from point cloud, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.

PDF Abstract

Results from the Paper


Ranked #2 on Semantic Segmentation on S3DIS Area5 (Number of params metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
3D Semantic Segmentation DALES PointCNN mIoU 58.4 # 8
Overall Accuracy 97.2 # 3
Model size N/A # 1
Few-Shot 3D Point Cloud Classification ModelNet40 10-way (10-shot) PointCNN Overall Accuracy 46.60 # 26
Standard Deviation 4.8 # 20
Few-Shot 3D Point Cloud Classification ModelNet40 10-way (20-shot) PointCNN Overall Accuracy 49.95 # 26
Standard Deviation 7.2 # 26
Few-Shot 3D Point Cloud Classification ModelNet40 5-way (20-shot) PointCNN Overall Accuracy 68.64 # 24
Standard Deviation 7.0 # 22
Semantic Segmentation S3DIS Area5 PointCNN oAcc 85.9 # 33
Number of params N/A # 2
Semantic Segmentation ScanNet PointCNN test mIoU 45.8 # 27

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
3D Point Cloud Classification IntrA PointCNN F1 score (5-fold) 0.875 # 6
3D Point Cloud Classification ModelNet40 PointCNN Overall Accuracy 92.2 # 87

Methods