Track to Detect and Segment: An Online Multi-Object Tracker

Most online multi-object trackers perform object detection stand-alone in a neural net without any input from tracking. In this paper, we present a new online joint detection and tracking model, TraDeS (TRAck to DEtect and Segment), exploiting tracking clues to assist detection end-to-end... TraDeS infers object tracking offset by a cost volume, which is used to propagate previous object features for improving current object detection and segmentation. Effectiveness and superiority of TraDeS are shown on 4 datasets, including MOT (2D tracking), nuScenes (3D tracking), MOTS and Youtube-VIS (instance segmentation tracking). Project page: read more

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Instance Segmentation Cityscapes test Average Precision 20.2 # 9
Multi-Object Tracking MOT15 Baseline+MFW MOTA 66.5 # 1
Online Multi-Object Tracking MOT16 TraDeS MOTA 67.7 # 1
Multi-Object Tracking MOT16 TraDeS MOTA 70.1 # 4
IDF1 64.7 # 5
Multi-Object Tracking MOT17 TraDeS MOTA 69.1 # 3
IDF1 63.9 # 6
Instance Segmentation nuScenes TraDeS MOTA 68.2 # 1
Video Instance Segmentation YouTube-VIS validation TraDeS mask AP 32.6 # 11
AP50 52.6 # 12
AP75 32.8 # 14