A Compositional Approach to Occlusion in Panoptic Segmentation

This paper concerns image segmentation, with emphasis on correctly classifying objects that are partially occluded. We present a novel approach based on compositional modeling that has proven to be effective at classifying separate instances of foreground objects. We demonstrate the efficacy of the approach by replacing the object detection pipeline in UPSNet with a compositional element that utilizes a mixture of distributions to model parts of objects. We also show extensive experimental results for the COCO and Cityscapes datasets. The results show an improvement of 2.6 points in panoptic quality for the top “thing” classes of COCO, and a 3.43% increase in overall recall, using standard UPSNet as a baseline. Moreover, we present qualitative results to demonstrate that improved metrics and datasets are needed for proper characterization of panoptic segmentation systems.