EmbedMask: Embedding Coupling for One-stage Instance Segmentation

4 Dec 2019  ·  Hui Ying, Zhaojin Huang, Shu Liu, Tianjia Shao, Kun Zhou ·

Current instance segmentation methods can be categorized into segmentation-based methods that segment first then do clustering, and proposal-based methods that detect first then predict masks for each instance proposal using repooling. In this work, we propose a one-stage method, named EmbedMask, that unifies both methods by taking advantages of them. Like proposal-based methods, EmbedMask builds on top of detection models making it strong in detection capability. Meanwhile, EmbedMask applies extra embedding modules to generate embeddings for pixels and proposals, where pixel embeddings are guided by proposal embeddings if they belong to the same instance. Through this embedding coupling process, pixels are assigned to the mask of the proposal if their embeddings are similar. The pixel-level clustering enables EmbedMask to generate high-resolution masks without missing details from repooling, and the existence of proposal embedding simplifies and strengthens the clustering procedure to achieve high speed with higher performance than segmentation-based methods. Without any bells and whistles, EmbedMask achieves comparable performance as Mask R-CNN, which is the representative two-stage method, and can produce more detailed masks at a higher speed. Code is available at github.com/yinghdb/EmbedMask.

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Instance Segmentation COCO test-dev EmbedMask (ResNet-101-FPN) mask AP 37.7% # 85
AP50 59.1% # 32
AP75 40.3% # 26
APS 17.9% # 28
APM 40.4% # 27
APL 53% # 23
Instance Segmentation COCO test-dev EmbedMask(R-101-FPN) mask AP 37.7 # 85
AP50 59.1 # 32
AP75 40.3 # 26
APS 17.9 # 28
APM 40.4 # 27