Point Transformer V2: Grouped Vector Attention and Partition-based Pooling

11 Oct 2022  ·  Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao ·

As a pioneering work exploring transformer architecture for 3D point cloud understanding, Point Transformer achieves impressive results on multiple highly competitive benchmarks. In this work, we analyze the limitations of the Point Transformer and propose our powerful and efficient Point Transformer V2 model with novel designs that overcome the limitations of previous work. In particular, we first propose group vector attention, which is more effective than the previous version of vector attention. Inheriting the advantages of both learnable weight encoding and multi-head attention, we present a highly effective implementation of grouped vector attention with a novel grouped weight encoding layer. We also strengthen the position information for attention by an additional position encoding multiplier. Furthermore, we design novel and lightweight partition-based pooling methods which enable better spatial alignment and more efficient sampling. Extensive experiments show that our model achieves better performance than its predecessor and achieves state-of-the-art on several challenging 3D point cloud understanding benchmarks, including 3D point cloud segmentation on ScanNet v2 and S3DIS and 3D point cloud classification on ModelNet40. Our code will be available at https://github.com/Gofinge/PointTransformerV2.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Point Cloud Classification ModelNet40 PTv2 Overall Accuracy 94.2 # 14
Mean Accuracy 91.6 # 8
LIDAR Semantic Segmentation nuScenes PTv2 test mIoU 0.826 # 2
val mIoU 0.802 # 2
3D Semantic Segmentation nuScenes PTv2 mIoU 82.6% # 1
3D Semantic Segmentation S3DIS PointTransformerV2 mIoU (Area-5) 71.6 # 2
Semantic Segmentation S3DIS Area5 PTv2 mIoU 72.6 # 10
oAcc 91.6 # 8
mAcc 78.0 # 14
Number of params N/A # 2
Semantic Segmentation ScanNet PTv2 test mIoU 75.2 # 10
val mIoU 75.4 # 10
3D Semantic Segmentation SemanticKITTI PTv2 test mIoU 72.6% # 7
val mIoU 70.3% # 6