Efficient Video Object Segmentation via Network Modulation
Video object segmentation targets at segmenting a specific object throughout a video sequence, given only an annotated first frame. Recent deep learning based approaches find it effective by fine-tuning a general-purpose segmentation model on the annotated frame using hundreds of iterations of gradient descent. Despite the high accuracy these methods achieve, the fine-tuning process is inefficient and fail to meet the requirements of real world applications. We propose a novel approach that uses a single forward pass to adapt the segmentation model to the appearance of a specific object. Specifically, a second meta neural network named modulator is learned to manipulate the intermediate layers of the segmentation network given limited visual and spatial information of the target object. The experiments show that our approach is 70times faster than fine-tuning approaches while achieving similar accuracy.
PDF Abstract CVPR 2018 PDF CVPR 2018 AbstractCode
ImageNet
MS COCO
DAVIS
DAVIS 2017
DAVIS 2016
YouTube-VOS 2018
YouTube-VIS 2019
Referring Expressions for DAVIS 2016 & 2017
Results from the Paper
Ranked #1 on
One-shot visual object segmentation
on YouTube-VOS 2018
(Jaccard (Seen) metric)
