PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation

24 Jul 2018  ·  Jonathon Luiten, Paul Voigtlaender, Bastian Leibe ·

We address semi-supervised video object segmentation, the task of automatically generating accurate and consistent pixel masks for objects in a video sequence, given the first-frame ground truth annotations. Towards this goal, we present the PReMVOS algorithm (Proposal-generation, Refinement and Merging for Video Object Segmentation). Our method separates this problem into two steps, first generating a set of accurate object segmentation mask proposals for each video frame and then selecting and merging these proposals into accurate and temporally consistent pixel-wise object tracks over a video sequence in a way which is designed to specifically tackle the difficult challenges involved with segmenting multiple objects across a video sequence. Our approach surpasses all previous state-of-the-art results on the DAVIS 2017 video object segmentation benchmark with a J & F mean score of 71.6 on the test-dev dataset, and achieves first place in both the DAVIS 2018 Video Object Segmentation Challenge and the YouTube-VOS 1st Large-scale Video Object Segmentation Challenge.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Video Object Segmentation DAVIS 2016 PReMVOS Jaccard (Mean) 84.9 # 47
Jaccard (Recall) 96.1 # 11
Jaccard (Decay) 8.8 # 17
F-measure (Mean) 88.6 # 38
F-measure (Recall) 94.7 # 9
F-measure (Decay) 9.8 # 13
J&F 86.75 # 41
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) PReMVOS J&F 71.6 # 35
Jaccard (Mean) 67.5 # 35
Jaccard (Recall) 76.8 # 4
Jaccard (Decay) 21.7 # 11
F-measure (Mean) 75.8 # 33
F-measure (Recall) 84.3 # 3
F-measure (Decay) 20.6 # 12
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) PReMVOS Jaccard (Mean) 73.9 # 40
Jaccard (Recall) 83.1 # 7
Jaccard (Decay) 16.2 # 12
F-measure (Mean) 81.8 # 35
F-measure (Recall) 88.9 # 5
F-measure (Decay) 19.5 # 11
J&F 77.85 # 36

Methods


No methods listed for this paper. Add relevant methods here