A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images
The fully convolutional network (FCN) with an encoder-decoder architecture has been the standard paradigm for semantic segmentation. The encoder-decoder architecture utilizes an encoder to capture multilevel feature maps, which are incorporated into the final prediction by a decoder. As the context is crucial for precise segmentation, tremendous effort has been made to extract such information in an intelligent fashion, including employing dilated/atrous convolutions or inserting attention modules. However, these endeavors are all based on the FCN architecture with ResNet or other backbones, which cannot fully exploit the context from the theoretical concept. By contrast, we introduce the Swin Transformer as the backbone to extract the context information and design a novel decoder of densely connected feature aggregation module (DCFAM) to restore the resolution and produce the segmentation map. The experimental results on two remotely sensed semantic segmentation datasets demonstrate the effectiveness of the proposed scheme.Code is available at https://github.com/WangLibo1995/GeoSeg
PDF AbstractCode
Datasets
ISPRS Potsdam
ISPRS Vaihingen
Results from the Paper
Ranked #3 on
Semantic Segmentation
on ISPRS Potsdam
(using extra training data)
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Semantic Segmentation | ISPRS Potsdam | DC-Swin | Overall Accuracy | 92.0 | # 3 | ||
| Mean F1 | 93.25 | # 4 | |||||
| Mean IoU | 87.56 | # 2 | |||||
| Semantic Segmentation | ISPRS Vaihingen | DC-Swin | Overall Accuracy | 91.6 | # 4 | ||
| Average F1 | 90.7 | # 5 |
