Scale Equivariance Improves Siamese Tracking

17 Jul 2020  ·  Ivan Sosnovik, Artem Moskalev, Arnold Smeulders ·

Siamese trackers turn tracking into similarity estimation between a template and the candidate regions in the frame. Mathematically, one of the key ingredients of success of the similarity function is translation equivariance. Non-translation-equivariant architectures induce a positional bias during training, so the location of the target will be hard to recover from the feature space. In real life scenarios, objects undergoe various transformations other than translation, such as rotation or scaling. Unless the model has an internal mechanism to handle them, the similarity may degrade. In this paper, we focus on scaling and we aim to equip the Siamese network with additional built-in scale equivariance to capture the natural variations of the target a priori. We develop the theory for scale-equivariant Siamese trackers, and provide a simple recipe for how to make a wide range of existing trackers scale-equivariant. We present SE-SiamFC, a scale-equivariant variant of SiamFC built according to the recipe. We conduct experiments on OTB and VOT benchmarks and on the synthetically generated T-MNIST and S-MNIST datasets. We demonstrate that a built-in additional scale equivariance is useful for visual object tracking.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Visual Object Tracking OTB-2013 SE-SiamFC AUC 0.68 # 1
Visual Object Tracking OTB-2015 SE-SiamFC AUC 0.66 # 10
Visual Object Tracking VOT2016 SE-SiamFC Expected Average Overlap (EAO) 0.36 # 4
Visual Object Tracking VOT2017 SE-SiamFC Expected Average Overlap (EAO) 0.27 # 4