Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers

CVPR 2021  ·  Lei Ke, Yu-Wing Tai, Chi-Keung Tang ·

Segmenting highly-overlapping objects is challenging, because typically no distinction is made between real object contours and occlusion boundaries. Unlike previous two-stage instance segmentation methods, we model image formation as composition of two overlapping layers, and propose Bilayer Convolutional Network (BCNet), where the top GCN layer detects the occluding objects (occluder) and the bottom GCN layer infers partially occluded instance (occludee). The explicit modeling of occlusion relationship with bilayer structure naturally decouples the boundaries of both the occluding and occluded instances, and considers the interaction between them during mask regression. We validate the efficacy of bilayer decoupling on both one-stage and two-stage object detectors with different backbones and network layer choices. Despite its simplicity, extensive experiments on COCO and KINS show that our occlusion-aware BCNet achieves large and consistent performance gain especially for heavy occlusion cases. Code is available at

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Instance Segmentation COCO test-dev BCNet(ResNeXt-101 + FPN+ FCOS) mask AP 41.7 # 53
Instance Segmentation COCO test-dev BCNet(ResNet-101-FPN + Faster RCNN) mask AP 39.8 # 69
AP50 61.5 # 23
AP75 43.1 # 19
APS 22.7 # 14
APM 42.4 # 20
APL 51.1 # 28
Instance Segmentation COCO test-dev BCNet(ResNet-101-FPN + FCOS) mask AP 39.6 # 72
AP50 61.2 # 26
AP75 42.7 # 22
APS 22.3 # 16
APM 42.3 # 21
APL 51.0 # 29
Instance Segmentation KINS BCNet mAP 28.87 # 1
Amodal Instance Segmentation WALT BCNet AP 73.2 # 2