Siam R-CNN: Visual Tracking by Re-Detection

We present Siam R-CNN, a Siamese re-detection architecture which unleashes the full power of two-stage object detection approaches for visual object tracking. We combine this with a novel tracklet-based dynamic programming algorithm, which takes advantage of re-detections of both the first-frame template and previous-frame predictions, to model the full history of both the object to be tracked and potential distractor objects. This enables our approach to make better tracking decisions, as well as to re-detect tracked objects after long occlusion. Finally, we propose a novel hard example mining strategy to improve Siam R-CNN's robustness to similar looking objects. Siam R-CNN achieves the current best performance on ten tracking benchmarks, with especially strong results for long-term tracking. We make our code and models available at www.vision.rwth-aachen.de/page/siamrcnn.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Object Tracking COESOT SiamR-CNN Success Rate 60.9 # 5
Precision Rate 67.5 # 6
Semi-Supervised Video Object Segmentation DAVIS 2016 Siam R-CNN Jaccard (Mean) 76.8 # 65
Jaccard (Recall) 86.4 # 28
Jaccard (Decay) 2.2 # 34
F-measure (Mean) 80.4 # 61
F-measure (Recall) 87.6 # 21
F-measure (Decay) 4.0 # 32
J&F 78.6 # 63
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) Siam R-CNN J&F 53.3 # 49
Jaccard (Mean) 48.0 # 52
Jaccard (Recall) 53.9 # 16
Jaccard (Decay) 21.8 # 12
F-measure (Mean) 58.6 # 48
F-measure (Recall) 62.3 # 13
F-measure (Decay) 20.2 # 11
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) Siam R-CNN Jaccard (Mean) 66.1 # 59
Jaccard (Recall) 74.8 # 12
Jaccard (Decay) 15.8 # 11
F-measure (Mean) 75.0 # 54
F-measure (Recall) 82.8 # 11
F-measure (Decay) 16.2 # 7
J&F 70.55 # 59
Visual Object Tracking GOT-10k Siam R-CNN Average Overlap 64.9 # 21
Success Rate 0.5 72.8 # 21
Visual Object Tracking LaSOT Siam R-CNN AUC 64.8 # 25
Normalized Precision 72.2 # 21
Visual Object Tracking TrackingNet Siam R-CNN Precision 80.0 # 15
Normalized Precision 85.4 # 18
Accuracy 81.2 # 19

Methods


No methods listed for this paper. Add relevant methods here