A Transductive Approach for Video Object Segmentation

CVPR 2020  ·  Yizhuo Zhang, Zhirong Wu, Houwen Peng, Stephen Lin ·

Semi-supervised video object segmentation aims to separate a target object from a video sequence, given the mask in the first frame. Most of current prevailing methods utilize information from additional modules trained in other domains like optical flow and instance segmentation, and as a result they do not compete with other methods on common ground. To address this issue, we propose a simple yet strong transductive method, in which additional modules, datasets, and dedicated architectural designs are not needed. Our method takes a label propagation approach where pixel labels are passed forward based on feature similarity in an embedding space. Different from other propagation methods, ours diffuses temporal information in a holistic manner which take accounts of long-term object appearance. In addition, our method requires few additional computational overhead, and runs at a fast $\sim$37 fps speed. Our single model with a vanilla ResNet50 backbone achieves an overall score of 72.3 on the DAVIS 2017 validation set and 63.1 on the test set. This simple yet high performing and efficient method can serve as a solid baseline that facilitates future research. Code and models are available at \url{https://github.com/microsoft/transductive-vos.pytorch}.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) TVOS Jaccard (Mean) 69.9 # 53
F-measure (Mean) 74.7 # 55
J&F 72.3 # 55
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) TVOS FPS 37.0 # 4
D17 val (G) 72.3 # 15
D17 val (J) 69.9 # 15
D17 val (F) 74.7 # 14
D17 test (G) 63.1 # 2
D17 test (J) 58.8 # 4
D17 test (F) 67.4 # 2


No methods listed for this paper. Add relevant methods here