TransTrack: Multiple Object Tracking with Transformer

31 Dec 2020  ·  Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zehuan Yuan, Changhu Wang, Ping Luo ·

In this work, we propose TransTrack, a simple but efficient scheme to solve the multiple object tracking problems. TransTrack leverages the transformer architecture, which is an attention-based query-key mechanism. It applies object features from the previous frame as a query of the current frame and introduces a set of learned object queries to enable detecting new-coming objects. It builds up a novel joint-detection-and-tracking paradigm by accomplishing object detection and object association in a single shot, simplifying complicated multi-step settings in tracking-by-detection methods. On MOT17 and MOT20 benchmark, TransTrack achieves 74.5\% and 64.5\% MOTA, respectively, competitive to the state-of-the-art methods. We expect TransTrack to provide a novel perspective for multiple object tracking. The code is available at: \url{https://github.com/PeizeSun/TransTrack}.

PDF Abstract

Results from the Paper


Ranked #6 on Multi-Object Tracking on SportsMOT (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Multi-Object Tracking DanceTrack TransTrack HOTA 45.7 # 22
DetA 72.1 # 21
AssA 27.5 # 22
MOTA 83.0 # 22
IDF1 44.8 # 22
Multi-Object Tracking SportsMOT TransTrack HOTA 68.9 # 6
IDF1 71.5 # 7
AssA 57.5 # 7
MOTA 92.6 # 8
DetA 82.7 # 5

Methods