Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
In this paper, we propose a long-sequence modeling framework, named StreamPETR, for multi-view 3D object detection. Built upon the sparse query design in the PETR series, we systematically develop an object-centric temporal mechanism. The model is performed in an online manner and the long-term historical information is propagated through object queries frame by frame. Besides, we introduce a motion-aware layer normalization to model the movement of the objects. StreamPETR achieves significant performance improvements only with negligible computation cost, compared to the single-frame baseline. On the standard nuScenes benchmark, it is the first online multi-view method that achieves comparable performance (67.6% NDS & 65.3% AMOTA) with lidar-based methods. The lightweight version realizes 45.0% mAP and 31.7 FPS, outperforming the state-of-the-art method (SOLOFusion) by 2.3% mAP and 1.8x faster FPS. Code has been available at https://github.com/exiawsh/StreamPETR.git.
PDF Abstract ICCV 2023 PDF ICCV 2023 AbstractCode
Datasets
nuScenes
Waymo Open Dataset
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| 3D Object Detection | 3D Object Detection on Argoverse2 Camera Only | StreamPETR | Average mAP | 20.3 | # 2 | |
| 3D Multi-Object Tracking | nuScenes Camera Only | StreamPETR-Large | AMOTA | 65.3 | # 1 | |
| 3D Object Detection | nuScenes Camera Only | StreamPETR-Large | NDS | 67.6 | # 4 | |
| Future Frame | false | # 1 |
