PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional Neural Network (CNN) and PointNet-based set abstraction to learn more discriminative point cloud features. It takes advantages of efficient learning and high-quality proposals of the 3D voxel CNN and the flexible receptive fields of the PointNet-based networks. Specifically, the proposed framework summarizes the 3D scene with a 3D voxel CNN into a small set of keypoints via a novel voxel set abstraction module to save follow-up computations and also to encode representative scene features. Given the high-quality 3D proposals generated by the voxel CNN, the RoI-grid pooling is proposed to abstract proposal-specific features from the keypoints to the RoI-grid points via keypoint set abstraction with multiple receptive fields. Compared with conventional pooling operations, the RoI-grid feature points encode much richer context information for accurately estimating object confidences and locations. Extensive experiments on both the KITTI dataset and the Waymo Open dataset show that our proposed PV-RCNN surpasses state-of-the-art 3D detection methods with remarkable margins by using only point clouds. Code is available at https://github.com/open-mmlab/OpenPCDet.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Robust 3D Object Detection KITTI-C PV-RCNN mean Corruption Error (mCE) 90.04% # 1
3D Object Detection KITTI Cars Easy PV-RCNN AP 90.25% # 7
Birds Eye View Object Detection KITTI Cars Easy PV-RCNN AP 94.98 # 2
Birds Eye View Object Detection KITTI Cars Hard PV-RCNN AP 86.14 # 3
3D Object Detection KITTI Cars Hard PV-RCNN AP 76.82% # 7
Birds Eye View Object Detection KITTI Cars Moderate PV-RCNN AP 90.65% # 2
3D Object Detection KITTI Cars Moderate PV-RCNN AP 81.43% # 11
Birds Eye View Object Detection KITTI Cyclists Easy PV-RCNN AP 82.49 # 1
3D Object Detection KITTI Cyclists Easy PV-RCNN AP 78.60% # 6
3D Object Detection KITTI Cyclists Hard PV-RCNN AP 57.65% # 3
Birds Eye View Object Detection KITTI Cyclists Hard PV-RCNN AP 62.41 # 1
Birds Eye View Object Detection KITTI Cyclists Moderate PV-RCNN AP 68.89% # 1
3D Object Detection KITTI Cyclists Moderate PV-RCNN AP 63.71% # 5
3D Object Detection waymo all_ns PV-RCNN APH/L2 71.52 # 2
3D Object Detection waymo cyclist PV-RCNN APH/L2 71.16 # 5
3D Object Detection waymo pedestrian PV-RCNN APH/L2 70.16 # 6
3D Object Detection waymo vehicle PV-RCNN APH/L2 73.23 # 4

Methods


No methods listed for this paper. Add relevant methods here