MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers

We present MaX-DeepLab, the first end-to-end model for panoptic segmentation. Our approach simplifies the current pipeline that depends heavily on surrogate sub-tasks and hand-designed components, such as box detection, non-maximum suppression, thing-stuff merging, etc... (read more)

PDF Abstract

Datasets


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Panoptic Segmentation COCO panoptic MaX-DeepLab-L (single-scale) PQ 51.1 # 1
PQth 57.0 # 1
PQst 42.2 # 1
Panoptic Segmentation COCO test-dev MaX-DeepLab-L (single-scale) PQ 51.3 # 2
PQst 42.4 # 1
PQth 57.2 # 5

Methods used in the Paper


METHOD TYPE
Scaled Dot-Product Attention
Attention Mechanisms
Softmax
Output Functions
Dropout
Regularization
BPE
Subword Segmentation
Label Smoothing
Regularization
Multi-Head Attention
Attention Modules
Adam
Stochastic Optimization
Layer Normalization
Normalization
Convolution
Convolutions
Residual Connection
Skip Connections
Transformer
Transformers
Dense Connections
Feedforward Networks
Feedforward Network
Feedforward Networks
Detr
Object Detection Models