Intra-frame Object Tracking by Deblatting

9 May 2019  ·  Jan Kotera, Denys Rozumnyi, Filip Šroubek, Jiří Matas ·

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects elapse non-negligible distance during exposure time of a single frame and therefore their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by standard trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. The trajectory is then estimated by fitting a piecewise quadratic curve, which models physically justifiable trajectories. As a result, tracked objects are precisely localized with higher temporal resolution than by conventional trackers. The proposed TbD tracker was evaluated on a newly created dataset of videos with ground truth obtained by a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms baseline both in recall and trajectory accuracy.

PDF Abstract

Datasets


Introduced in the Paper:

TbD

Used in the Paper:

TbD-3D Falling Objects

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Video Super-Resolution Falling Objects TbD SSIM 0.591 # 3
PSNR 20.53 # 3
TIoU 0.539 # 1
Video Super-Resolution TbD TbD SSIM 0.605 # 2
PSNR 23.22 # 3
TIoU 0.542 # 2
Video Super-Resolution TbD-3D TbD SSIM 0.504 # 3
PSNR 18.84 # 3
TIoU 0.598 # 2

Methods