ID	33148034
Memory (M)	10466
Output Resolution	1024×2048

ID	246448865
LR	0.0005
Max Iter	200000
Backbone Layers	50
Output Resolution	640×640

Memory (M)	8668
Output Resolution	1024×2048

ID	30841561
Max Iter	900000
Memory (M)	9682
Backbone Layers	50
Output Resolution	1024×2048

Panoptic-DeepLab

facebookresearch / detectron2

Last updated on Feb 19, 2021

Parameters 30 Million

FLOPs

File Size 115.89 MB

Training Data Cityscapes

Training Resources 8 NVIDIA V100 GPUs

Training Time

Architecture	1x1 Convolution, Convolution, ASPP, Dilated Convolution
ID	33148034
Memory (M)	10466
Output Resolution	1024×2048
SHOW MORE
SHOW LESS

Parameters 30 Million

Backbone Layers 50

File Size 116.00 MB

Training Data MS COCO

Training Resources 8 NVIDIA V100 GPUs

Training Time

Architecture	1x1 Convolution, Convolution, ASPP, Dilated Convolution
ID	246448865
LR	0.0005
Max Iter	200000
Backbone Layers	50
Output Resolution	640×640
SHOW MORE
SHOW LESS

Memory (M) 8668

FLOPs

Training Data Cityscapes

Training Resources 8 NVIDIA V100 GPUs

Training Time

Architecture	1x1 Convolution, Convolution, ASPP, ResNet, Dilated Convolution
Memory (M)	8668
Output Resolution	1024×2048
SHOW MORE
SHOW LESS

Parameters 60 Million

Backbone Layers 50

File Size 228.31 MB

Training Data Cityscapes

Training Resources 8 NVIDIA V100 GPUs

Training Time

Architecture	1x1 Convolution, Convolution, ASPP, Dilated Convolution
ID	30841561
Max Iter	900000
Memory (M)	9682
Backbone Layers	50
Output Resolution	1024×2048
SHOW MORE
SHOW LESS

README.md

Summary

Panoptic-DeepLab is a panoptic segmentation architecture. In particular, Panoptic-DeepLab adopts the dual-ASPP and dual-decoder structures specific to semantic, and instance segmentation, respectively. The semantic segmentation branch is the same as the typical design of any semantic segmentation model (e.g., DeepLab), while the instance segmentation branch is class-agnostic, involving a simple instance center regression.

Installation

Install Detectron2 following the instructions.

To use Cityscapes, prepare data follow the tutorial.

Training

To train a model with 8 GPUs run:

cd /path/to/detectron2/projects/Panoptic-DeepLab
python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --num-gpus 8

Evaluation

Model evaluation can be done similarly:

cd /path/to/detectron2/projects/Panoptic-DeepLab
python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Benchmark network speed

If you want to benchmark the network speed without post-processing, you can run the evaluation script with MODEL.PANOPTIC_DEEPLAB.BENCHMARK_NETWORK_SPEED True:

cd /path/to/detectron2/projects/Panoptic-DeepLab
python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --eval-only MODEL.WEIGHTS /path/to/model_checkpoint MODEL.PANOPTIC_DEEPLAB.BENCHMARK_NETWORK_SPEED True

Citation

@inproceedings{cheng2020panoptic,
  title={Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation},
  author={Cheng, Bowen and Collins, Maxwell D and Zhu, Yukun and Liu, Ting and Huang, Thomas S and Adam, Hartwig and Chen, Liang-Chieh},
  booktitle={CVPR},
  year={2020}
}

Results

Panoptic Segmentation on Cityscapes val

Panoptic Segmentation

BENCHMARK	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Cityscapes val	Panoptic-DeepLab (DSConv) (R52-DC5)	PQ	60.3	# 1
		mIoU	78.7	# 1
		AP	32.1	# 1
		SQ	81.0	# 1
		RQ	73.2	# 1
Cityscapes val	Panoptic-DeepLab (R52-DC5)	PQ	60.3	# 1
		mIoU	78.2	# 1
		AP	33.2	# 1
		SQ	81.5	# 1
		RQ	72.9	# 1
Cityscapes val	Panoptic-DeepLab (R50-DC5)	PQ	58.6	# 2
		mIoU	75.9	# 2
		AP	29.8	# 2
		SQ	80.9	# 2
		RQ	71.2	# 2
COCO minival	Panoptic-DeepLab (DSConv) (R52-DC5, COCO)	PQ	35.5	# 4
		SQ	77.3	# 4
		RQ	44.7	# 4