Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation

Convolution exploits locality for efficiency at a cost of missing long range context. Self-attention has been adopted to augment CNNs with non-local interactions... Recent works prove it possible to stack self-attention layers to obtain a fully attentional network by restricting the attention to a local region. In this paper, we attempt to remove this constraint by factorizing 2D self-attention into two 1D self-attentions. This reduces computation complexity and allows performing attention within a larger or even global region. In companion, we also propose a position-sensitive self-attention design. Combining both yields our position-sensitive axial-attention layer, a novel building block that one could stack to form axial-attention models for image classification and dense prediction. We demonstrate the effectiveness of our model on four large-scale datasets. In particular, our model outperforms all existing stand-alone self-attention models on ImageNet. Our Axial-DeepLab improves 2.8% PQ over bottom-up state-of-the-art on COCO test-dev. This previous state-of-the-art is attained by our small variant that is 3.8x parameter-efficient and 27x computation-efficient. Axial-DeepLab also achieves state-of-the-art results on Mapillary Vistas and Cityscapes. read more

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Panoptic Segmentation Cityscapes test Axial-DeepLab-XL (Mapillary Vistas, multi-scale) PQ 66.6 # 3
Panoptic Segmentation Cityscapes val Axial-DeepLab-XL (Mapillary Vistas, multi-scale) PQ 68.5 # 2
mIoU 84.6 # 3
AP 44.2 # 2
Panoptic Segmentation COCO minival Axial-DeepLab-L(multi-scale) PQth 48.6 # 7
PQst 36.8 # 5
Panoptic Segmentation COCO minival Axial-DeepLab-L (single-scale) PQ 43.4 # 8
PQth 48.5 # 8
PQst 35.6 # 6
Panoptic Segmentation COCO minival Axial-DeepLab-L (multi-scale) PQ 43.9 # 7
Panoptic Segmentation COCO test-dev Axial-DeepLab-L (multi-scale) PQ 44.2 # 18
PQst 36.8 # 12
PQth 49.2 # 18
Panoptic Segmentation COCO test-dev Axial-DeepLab-L PQ 43.6 # 19
PQst 35.6 # 16
PQth 48.9 # 19
Panoptic Segmentation Mapillary val Axial-DeepLab-L (multi-scale) PQ 41.1 # 2
mIoU 58.4 # 2
PQth 33.4 # 2
PQst 51.3 # 2


No methods listed for this paper. Add relevant methods here