PCT: Point cloud transformer

17 Dec 2020  ·  Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R. Martin, Shi-Min Hu ·

The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation and normal estimation tasks.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Point Cloud Classification IntrA PCT F1 score (5-fold) 0.914 # 2
3D Point Cloud Classification ModelNet40 Point Cloud Transformer Overall Accuracy 93.2 # 60
Number of params 2.88M # 95
3D Point Cloud Classification ModelNet40-C PCT Error Rate 0.255 # 8
Point Cloud Classification PointCloud-C PCT mean Corruption Error (mCE) 0.925 # 14
3D Part Segmentation ShapeNet-Part Point Cloud Transformer Instance Average IoU 86.4 # 22