PointCNN: Convolution On X-Transformed Points

We present a simple and general framework for feature learning from point cloud. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point cloud are irregular and unordered, thus a direct convolving of kernels against the features associated with the points will result in deserting the shape information while being variant to the orders. To address these problems, we propose to learn a X-transformation from the input points, which is used for simultaneously weighting the input features associated with the points and permuting them into latent potentially canonical order. Then element-wise product and sum operations of typical convolution operator are applied on the X-transformed features. The proposed method is a generalization of typical CNNs into learning features from point cloud, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Semantic Segmentation S3DIS Area5 PointCNN oAcc 85.9 # 12
Semantic Segmentation ScanNet PointCNN 3DIoU 0.458 # 15

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
3D Point Cloud Classification IntrA PointCNN F1 score (5-fold) 0.875 # 6
3D Point Cloud Classification ModelNet40 PointCNN Overall Accuracy 92.2 # 54