One-Shot Video Object Segmentation

This paper tackles the task of semi-supervised video object segmentation, i.e., the separation of an object from the background in a video, given the mask of the first frame. We present One-Shot Video Object Segmentation (OSVOS), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one-shot). Although all frames are processed independently, the results are temporally coherent and stable. We perform experiments on two annotated video segmentation databases, which show that OSVOS is fast and improves the state of the art by a significant margin (79.8% vs 68.0%).

PDF Abstract CVPR 2017 PDF CVPR 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Video Object Segmentation DAVIS 2016 OSVOS Jaccard (Mean) 79.8 # 59
Jaccard (Recall) 93.6 # 17
Jaccard (Decay) 14.9 # 5
F-measure (Mean) 80.6 # 57
F-measure (Recall) 92.6 # 12
F-measure (Decay) 15.0 # 4
J&F 80.2 # 59
Semi-Supervised Video Object Segmentation DAVIS 2017 (test-dev) OSVOS J&F 50.9 # 50
Jaccard (Mean) 47.0 # 51
Jaccard (Recall) 52.1 # 18
Jaccard (Decay) 19.2 # 8
F-measure (Recall) 59.7 # 18
F-measure (Decay) 19.8 # 8
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) OSVOS Jaccard (Mean) 56.6 # 64
Jaccard (Recall) 63.8 # 21
Jaccard (Decay) 26.1 # 22
F-measure (Mean) 63.9 # 62
F-measure (Recall) 73.8 # 18
F-measure (Decay) 27.0 # 20
J&F 60.25 # 64
One-shot visual object segmentation YouTube-VOS 2018 OSVOS F-Measure (Seen) 60.5 # 1
Semi-Supervised Video Object Segmentation YouTube-VOS 2018 OSVOS F-Measure (Seen) 60.5 # 50
F-Measure (Unseen) 60.7 # 47
Overall 58.8 # 49
Speed (FPS) 0.10 # 26
Jaccard (Seen) 59.8 # 50
Jaccard (Unseen) 54.2 # 44
Visual Object Tracking YouTube-VOS 2018 OSVOS O (Average of Measures) 58.8 # 1
F-Measure (Seen) 60.5 # 3
F-Measure (Unseen) 60.7 # 3

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Semi-Supervised Video Object Segmentation YouTube OSVOS mIoU 0.783 # 4

Methods


No methods listed for this paper. Add relevant methods here