Learning Video Object Segmentation from Unlabeled Videos

We propose a new method for video object segmentation (VOS) that addresses object pattern learning from unlabeled videos, unlike most existing methods which rely heavily on extensive annotated data. We introduce a unified unsupervised/weakly supervised learning framework, called MuG, that comprehensively captures intrinsic properties of VOS at multiple granularities. Our approach can help advance understanding of visual patterns in VOS and significantly reduce annotation burden. With a carefully-designed architecture and strong representation learning ability, our learned model can be applied to diverse VOS settings, including object-level zero-shot VOS, instance-level zero-shot VOS, and one-shot VOS. Experiments demonstrate promising performance in these settings, as well as the potential of MuG in leveraging unlabeled data to further improve the segmentation accuracy.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Unsupervised Video Object Segmentation DAVIS 2016 MuG-W Jaccard (Mean) 62.0 # 24
Jaccard (Recall) 74.7 # 22
Jaccard (Decay) -3.1 # 24
F-measure (Mean) 56.1 # 24
F-measure (Recall) 62.1 # 23
F-measure (Decay) -3.6 # 23
J&F 59.05 # 26
Semi-Supervised Video Object Segmentation DAVIS 2016 MuG-W Jaccard (Mean) 65.7 # 62
Jaccard (Recall) 77.7 # 31
Jaccard (Decay) 26.4 # 2
F-measure (Mean) 63.6 # 59
F-measure (Recall) 67.7 # 33
F-measure (Decay) 27.2 # 1
J&F 64.65 # 61
Unsupervised Video Object Segmentation DAVIS 2017 (test-dev) MuG-W J&F 41.7 # 3
Jaccard (Mean) 38.9 # 3
Jaccard (Recall) 44.3 # 3
Jaccard (Decay) -2.7 # 5
F-measure (Mean) 44.5 # 3
F-measure (Recall) 46.6 # 3
F-measure (Decay) -1.7 # 5
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) MuG-W Jaccard (Mean) 54.1 # 56
Jaccard (Recall) 60.5 # 25
Jaccard (Decay) 32.5 # 24
F-measure (Mean) 58.0 # 56
F-measure (Recall) 62.2 # 25
F-measure (Decay) 37.4 # 22
J&F 56.05 # 56

Methods


No methods listed for this paper. Add relevant methods here