Center-based 3D Object Detection and Tracking

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, CenterPoint outperforms all previous single model method by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Multi-Object Tracking nuScenes CenterPoint-Single amota 0.64 # 32
3D Object Detection nuScenes CenterPoint NDS 0.71 # 16
mAP 0.67 # 15
mATE 0.25 # 153
mASE 0.24 # 112
mAOE 0.35 # 130
mAVE 0.25 # 134
mAAE 0.14 # 59
3D Object Detection waymo all_ns CenterPoint APH/L2 71.93 # 1
3D Object Detection waymo cyclist CenterPoint APH/L2 71.28 # 2
3D Object Detection waymo pedestrian CenterPoint APH/L2 71.52 # 2