ByteTrack: Multi-Object Tracking by Associating Every Detection Box

Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects in videos. Most methods obtain identities by associating detection boxes whose scores are higher than a threshold. The objects with low detection scores, e.g. occluded objects, are simply thrown away, which brings non-negligible true object missing and fragmented trajectories. To solve this problem, we present a simple, effective and generic association method, tracking by associating almost every detection box instead of only the high score ones. For the low score detection boxes, we utilize their similarities with tracklets to recover true objects and filter out the background detections. When applied to 9 different state-of-the-art trackers, our method achieves consistent improvement on IDF1 score ranging from 1 to 10 points. To put forwards the state-of-the-art performance of MOT, we design a simple and strong tracker, named ByteTrack. For the first time, we achieve 80.3 MOTA, 77.3 IDF1 and 63.1 HOTA on the test set of MOT17 with 30 FPS running speed on a single V100 GPU. ByteTrack also achieves state-of-the-art performance on MOT20, HiEve and BDD100K tracking benchmarks. The source code, pre-trained models with deploy versions and tutorials of applying to other trackers are released at https://github.com/ifzhang/ByteTrack.

PDF Abstract arXiv 2021 PDF arXiv 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Multiple Object Tracking BDD100K test ByteTrack mMOTA 40.1 # 3
mIDF1 55.8 # 3
Multiple Object Tracking BDD100K val ByteTrack mMOTA 45.5 # 1
mIDF1 54.8 # 4
TETA 55.7 # 1
AssocA 51.5 # 3
Multi-Object Tracking DanceTrack ByteTrack HOTA 47.1 # 25
DetA 70.5 # 26
AssA 31.5 # 23
MOTA 88.2 # 21
IDF1 51.9 # 23
Multi-Object Tracking MOT17 ByteTrack MOTA 80.3 # 8
IDF1 77.3 # 13
HOTA 63.1 # 11
Multi-Object Tracking MOT20 ByteTrack MOTA 77.8 # 3
IDF1 75.2 # 11
HOTA 61.3 # 10
Multi-Object Tracking SportsMOT ByteTrack HOTA 64.1 # 14
IDF1 71.4 # 12
AssA 52.3 # 14
MOTA 95.9 # 11
DetA 78.5 # 15
Multiple Object Tracking SportsMOT ByteTrack HOTA 64.1 # 13
IDF1 71.4 # 11
AssA 52.3 # 13
MOTA 95.9 # 10
DetA 78.5 # 14

Methods