Center-based 3D Object Detection and Tracking

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, CenterPoint outperforms all previous single model method by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Multi-Object Tracking nuScenes CenterPoint-Single amota 0.64 # 32
3D Object Detection nuScenes CenterPoint NDS 0.71 # 16
mAP 0.67 # 15
mATE 0.25 # 153
mASE 0.24 # 112
mAOE 0.35 # 130
mAVE 0.25 # 134
mAAE 0.14 # 59
3D Object Detection waymo all_ns CenterPoint APH/L2 71.93 # 1
3D Object Detection waymo cyclist CenterPoint APH/L2 71.28 # 2
3D Object Detection waymo pedestrian CenterPoint APH/L2 71.52 # 2

Methods