Panoptic Segmentation
213 papers with code • 24 benchmarks • 32 datasets
Panoptic Segmentation is a computer vision task that combines semantic segmentation and instance segmentation to provide a comprehensive understanding of the scene. The goal of panoptic segmentation is to segment the image into semantically meaningful parts or regions, while also detecting and distinguishing individual instances of objects within those regions. In a given image, every pixel is assigned a semantic label, and pixels belonging to "things" classes (countable objects with instances, like cars and people) are assigned unique instance IDs. ( Image credit: Detectron2 )
Benchmarks
These leaderboards are used to track progress in Panoptic Segmentation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Panoptic Segmentation models and implementations
MS COCO
Cityscapes
KITTI
ScanNet
ADE20K
NYUv2
SemanticKITTI
SUN RGB-D
S3DIS
BDD100K
Latest papers
ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning
Panoptic segmentation, combining semantic and instance segmentation, stands as a cutting-edge computer vision task.
PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model
zamling/psalm
•
•
PSALM is a powerful extension of the Large Multi-modal Model (LMM) to address the segmentation task challenges.
PosSAM: Panoptic Open-vocabulary Segment Anything
Vibashan/PosSAM
•
•
In this paper, we introduce an open-vocabulary panoptic segmentation model that effectively unifies the strengths of the Segment Anything Model (SAM) with the vision-language CLIP model in an end-to-end framework.
PEM: Prototype-based Efficient MaskFormer for Image Segmentation
To fill this gap, we propose Prototype-based Efficient MaskFormer (PEM), an efficient transformer-based architecture that can operate in multiple segmentation tasks.
Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review
This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers.
OMG-Seg: Is One Model Good Enough For All Segmentation?
lxtgh/omg-seg
•
•
In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models.
RAP-SAM: Towards Real-Time All-Purpose Segment Anything
Segment Anything Model (SAM) is one remarkable model that can achieve generalized segmentation.
A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting
segments-ai/latent-diffusion-segmentation
•
•
Panoptic and instance segmentation networks are often trained with specialized object detection modules, complex loss functions, and ad-hoc post-processing steps to handle the permutation-invariance of the instance masks.
Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering
drprojects/superpoint_transformer
•
•
We introduce a highly efficient method for panoptic segmentation of large 3D point clouds by redefining this task as a scalable graph clustering problem.
Unsupervised Universal Image Segmentation
u2seg/u2seg
•
•
Several unsupervised image segmentation approaches have been proposed which eliminate the need for dense manually-annotated segmentation masks; current models separately handle either semantic segmentation (e. g., STEGO) or class-agnostic instance segmentation (e. g., CutLER), but not both (i. e., panoptic segmentation).
