UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking

15 Jan 2020  ·  Jonathon Luiten, Idil Esen Zulfikar, Bastian Leibe ·

We address Unsupervised Video Object Segmentation (UVOS), the task of automatically generating accurate pixel masks for salient objects in a video sequence and of tracking these objects consistently through time, without any input about which objects should be tracked. Towards solving this task, we present UnOVOST (Unsupervised Offline Video Object Segmentation and Tracking) as a simple and generic algorithm which is able to track and segment a large variety of objects. This algorithm builds up tracks in a number stages, first grouping segments into short tracklets that are spatio-temporally consistent, before merging these tracklets into long-term consistent object tracks based on their visual similarity. In order to achieve this we introduce a novel tracklet-based Forest Path Cutting data association algorithm which builds up a decision forest of track hypotheses before cutting this forest into paths that form long-term consistent object tracks. When evaluating our approach on the DAVIS 2017 Unsupervised dataset we obtain state-of-the-art performance with a mean J &F score of 67.9% on the val, 58% on the test-dev and 56.4% on the test-challenge benchmarks, obtaining first place in the DAVIS 2019 Unsupervised Video Object Segmentation Challenge. UnOVOST even performs competitively with many semi-supervised video object segmentation algorithms even though it is not given any input as to which objects should be tracked and segmented.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Video Object Segmentation DAVIS 2017 (test-dev) UnOVSOT J&F 58.0 # 2
Jaccard (Mean) 54.0 # 1
Jaccard (Recall) 62.9 # 1
Jaccard (Decay) 3.5 # 3
F-measure (Mean) 62.0 # 1
F-measure (Recall) 66.6 # 1
F-measure (Decay) 6.6 # 4
Unsupervised Video Object Segmentation DAVIS 2017 (val) UnOVOST J&F 67.9 # 3
Jaccard (Mean) 66.4 # 3
Jaccard (Recall) 76.4 # 1
F-measure (Mean) 69.3 # 3
F-measure (Recall) 76.9 # 2

Methods


No methods listed for this paper. Add relevant methods here