Bridging the Gap Between End-to-end and Non-End-to-end Multi-Object Tracking

22 May 2023  ·  Feng Yan, Weixin Luo, Yujie Zhong, Yiyang Gan, Lin Ma ·

Existing end-to-end Multi-Object Tracking (e2e-MOT) methods have not surpassed non-end-to-end tracking-by-detection methods. One potential reason is its label assignment strategy during training that consistently binds the tracked objects with tracking queries and then assigns the few newborns to detection queries. With one-to-one bipartite matching, such an assignment will yield unbalanced training, i.e., scarce positive samples for detection queries, especially for an enclosed scene, as the majority of the newborns come on stage at the beginning of videos. Thus, e2e-MOT will be easier to yield a tracking terminal without renewal or re-initialization, compared to other tracking-by-detection methods. To alleviate this problem, we present Co-MOT, a simple and effective method to facilitate e2e-MOT by a novel coopetition label assignment with a shadow concept. Specifically, we add tracked objects to the matching targets for detection queries when performing the label assignment for training the intermediate decoders. For query initialization, we expand each query by a set of shadow counterparts with limited disturbance to itself. With extensive ablations, Co-MOT achieves superior performance without extra costs, e.g., 69.4% HOTA on DanceTrack and 52.8% TETA on BDD100K. Impressively, Co-MOT only requires 38\% FLOPs of MOTRv2 to attain a similar performance, resulting in the 1.4$\times$ faster inference speed.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Multi-Object Tracking BDD100K CO-MOT TETA 52.8 # 1
LocA 38.7 # 1
AssocA 56.2 # 1
ClsA 63.6 # 1
Multi-Object Tracking DanceTrack CO-MOT HOTA 69.4 # 2
DetA 82.1 # 5
AssA 58.9 # 2
MOTA 91.2 # 9
IDF1 71.9 # 2
Multi-Object Tracking MOT17 CO-MOT(ours) MOTA 72.6 # 18
IDF1 72.7 # 13
HOTA 60.1 # 12
DetA 59.5 # 1
AssA 60.6 # 1
Video Object Tracking SoccerNet-v2 CO-MOT HOTA 69.54 # 1


No methods listed for this paper. Add relevant methods here