KPConv: Flexible and Deformable Convolution for Point Clouds

We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Semantic Segmentation DALES KPConv mIoU 81.1 # 1
Overall Accuracy 97.8 # 1
Model size 14M # 7
3D Point Cloud Classification ModelNet40 KPConv Overall Accuracy 92.9 # 68
LIDAR Semantic Segmentation Paris-Lille-3D KPConv deform mIOU 0.759 # 4
Semantic Segmentation S3DIS KPConv Mean IoU 70.6 # 22
mAcc 79.1 # 19
Number of params 14.1M # 46
Params (M) 14.1 # 6
Semantic Segmentation S3DIS Area5 KPConv mIoU 67.1 # 34
mAcc 72.8 # 26
Number of params 14.1M # 50
Scene Segmentation ScanNet KPConv 3DIoU 68.6 # 1
Semantic Segmentation ScanNet KpConv test mIoU 68.0 # 19
val mIoU 69.2 # 19
3D Semantic Segmentation ScanNet++ KPConv Top-1 IoU 0.265 # 3
Semantic Segmentation Semantic3D KPConv mIoU 74.6% # 7
3D Semantic Segmentation SemanticKITTI KPConv test mIoU 58.8% # 21
Robust 3D Semantic Segmentation SemanticKITTI-C KPConv mean Corruption Error (mCE) 99.54% # 2
3D Semantic Segmentation SensatUrban KPConv mIoU 57.58 # 3
3D Part Segmentation ShapeNet-Part KPConv Class Average IoU 85.1 # 7
Instance Average IoU 86.4 # 22
3D Semantic Segmentation STPLS3D KpConv mIOU 53.73 # 1