Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation

Convolution exploits locality for efficiency at a cost of missing long range context. Self-attention has been adopted to augment CNNs with non-local interactions. Recent works prove it possible to stack self-attention layers to obtain a fully attentional network by restricting the attention to a local region. In this paper, we attempt to remove this constraint by factorizing 2D self-attention into two 1D self-attentions. This reduces computation complexity and allows performing attention within a larger or even global region. In companion, we also propose a position-sensitive self-attention design. Combining both yields our position-sensitive axial-attention layer, a novel building block that one could stack to form axial-attention models for image classification and dense prediction. We demonstrate the effectiveness of our model on four large-scale datasets. In particular, our model outperforms all existing stand-alone self-attention models on ImageNet. Our Axial-DeepLab improves 2.8% PQ over bottom-up state-of-the-art on COCO test-dev. This previous state-of-the-art is attained by our small variant that is 3.8x parameter-efficient and 27x computation-efficient. Axial-DeepLab also achieves state-of-the-art results on Mapillary Vistas and Cityscapes.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Results from the Paper


Ranked #4 on Panoptic Segmentation on Cityscapes val (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Panoptic Segmentation Cityscapes test Axial-DeepLab-XL (Mapillary Vistas, multi-scale) PQ 66.6 # 4
Panoptic Segmentation Cityscapes val Axial-DeepLab-XL (Mapillary Vistas, multi-scale) PQ 68.5 # 4
mIoU 84.6 # 3
AP 44.2 # 9
Panoptic Segmentation COCO minival Axial-DeepLab-L (multi-scale) PQ 43.9 # 24
Panoptic Segmentation COCO minival Axial-DeepLab-L (single-scale) PQ 43.4 # 25
PQth 48.5 # 22
PQst 35.6 # 19
Panoptic Segmentation COCO minival Axial-DeepLab-L(multi-scale) PQth 48.6 # 21
PQst 36.8 # 18
Panoptic Segmentation COCO test-dev Axial-DeepLab-L (multi-scale) PQ 44.2 # 28
PQst 36.8 # 20
PQth 49.2 # 26
Panoptic Segmentation COCO test-dev Axial-DeepLab-L PQ 43.6 # 29
PQst 35.6 # 24
PQth 48.9 # 27
Panoptic Segmentation Mapillary val Axial-DeepLab-L (multi-scale) PQ 41.1 # 6
mIoU 58.4 # 4
PQth 33.4 # 6
PQst 51.3 # 6

Methods


No methods listed for this paper. Add relevant methods here