FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling

17 Dec 2020  ·  Kangcheng Liu, Zhi Gao, Feng Lin, Ben M. Chen ·

This work presents FG-Net, a general deep learning framework for large-scale point clouds understanding without voxelizations, which achieves accurate and real-time performance with a single NVIDIA GTX 1080 GPU. First, a novel noise and outlier filtering method is designed to facilitate subsequent high-level tasks. For effective understanding purpose, we propose a deep convolutional neural network leveraging correlated feature mining and deformable convolution based geometric-aware modelling, in which the local feature relationships and geometric patterns can be fully exploited. For the efficiency issue, we put forward an inverse density sampling operation and a feature pyramid based residual learning strategy to save the computational cost and memory consumption respectively. Extensive experiments on real-world challenging datasets demonstrated that our approaches outperform state-of-the-art approaches in terms of accuracy and efficiency. Moreover, weakly supervised transfer learning is also conducted to demonstrate the generalization capacity of our method.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Point Cloud Classification ModelNet40 Feature Geometric Net (FG-Net) Overall Accuracy 93.8 # 15
Mean Accuracy 91.1 # 11
LIDAR Semantic Segmentation Paris-Lille-3D Feature Geometric Net (FG Net) mIOU 0.819 # 2
3D Semantic Segmentation PartNet FG-Net mIOU 58.2 # 2
Semantic Segmentation S3DIS Feature Geometric Net (FG-Net) Mean IoU 70.8 # 10
mAcc 82.9 # 5
oAcc 88.2 # 14
Semantic Segmentation ScanNet FG-Net 3DIoU 0.690 # 7
Semantic Segmentation Semantic3D Feature Geometric Net mIoU 78.2% # 1
oAcc 93.6 # 4
3D Semantic Segmentation SemanticKITTI FG-Net mIoU 53.8% # 20
3D Part Segmentation ShapeNet-Part Feature Geometric Net (FG-Net) Class Average IoU 87.7 # 1
Instance Average IoU 86.6 # 11