Panoptic segmentation unifies the typically distinct tasks of semantic segmentation (assign a class label to each pixel) and instance segmentation (detect and segment each object instance).
( Image credit: Detectron2 )
We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences and extra images to surpass state-of-the-art performance on core computer vision tasks.
OPTICAL FLOW ESTIMATION PANOPTIC SEGMENTATION PATCH MATCHING SCENE SEGMENTATION
Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance.
Ranked #1 on
Real-Time Object Detection
on COCO minival
(MAP metric)
3D INSTANCE SEGMENTATION HUMAN PART SEGMENTATION KEYPOINT DETECTION MULTI-HUMAN PARSING MULTI-PERSON POSE ESTIMATION MULTI-TISSUE NUCLEUS SEGMENTATION NUCLEAR SEGMENTATION PANOPTIC SEGMENTATION REAL-TIME OBJECT DETECTION
The semantic segmentation branch is the same as the typical design of any semantic segmentation model (e. g., DeepLab), while the instance segmentation branch is class-agnostic, involving a simple instance center regression.
In this work, we perform a detailed study of this minimally extended version of Mask R-CNN with FPN, which we refer to as Panoptic FPN, and show it is a robust and accurate baseline for both tasks.
Ranked #4 on
Panoptic Segmentation
on KITTI Panoptic Segmentation
In this paper, we explore this mechanism in the backbone design for object detection.
Ranked #3 on
Panoptic Segmentation
on COCO test-dev
INSTANCE SEGMENTATION OBJECT DETECTION PANOPTIC SEGMENTATION
We present a new method that views object detection as a direct set prediction problem.
Ranked #3 on
Panoptic Segmentation
on COCO panoptic
It is well known that featuremap attention and multi-path representation are important for visual recognition.
Ranked #2 on
Panoptic Segmentation
on COCO panoptic
IMAGE CLASSIFICATION INSTANCE SEGMENTATION OBJECT DETECTION PANOPTIC SEGMENTATION TRANSFER LEARNING
Although current deep learning methods have achieved impressive results for semantic segmentation, they incur high computational costs and have a huge number of parameters.
Importantly, we take one step further by dynamically learning the mask head of the object segmenter such that the mask head is conditioned on the location.
Ranked #11 on
Instance Segmentation
on COCO test-dev
INSTANCE SEGMENTATION OBJECT DETECTION PANOPTIC SEGMENTATION
Multi-scale inference is commonly used to improve the results of semantic segmentation.
Ranked #1 on
Semantic Segmentation
on Cityscapes test
(using extra training data)