SiamixFormer: A Siamese Transformer Network For Building Detection And Change Detection From Bi-Temporal Remote Sensing Images

1 Aug 2022  ·  Amir mohammadian, Foad Ghaderi ·

Building detection and change detection using remote sensing images can help urban and rescue planning. Moreover, they can be used for building damage assessment after natural disasters. Currently, most of the existing models for building detection use only one image (pre-disaster image) to detect buildings. This is based on the idea that post-disaster images reduce the model's performance because of presence of destroyed buildings. In this paper, we propose a siamese model, called SiamixFormer, which uses pre- and post-disaster images as input. Our model has two encoders and has a hierarchical transformer architecture. The output of each stage in both encoders is given to a temporal transformer for feature fusion in a way that query is generated from pre-disaster images and (key, value) is generated from post-disaster images. To this end, temporal features are also considered in feature fusion. Another advantage of using temporal transformers in feature fusion is that they can better maintain large receptive fields generated by transformer encoders compared with CNNs. Finally, the output of the temporal transformer is given to a simple MLP decoder at each stage. The SiamixFormer model is evaluated on xBD, and WHU datasets, for building detection and on LEVIR-CD and CDD datasets for change detection and could outperform the state-of-the-art.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-0 F1-Score 0.9205 # 17
IoU 85.81 # 7
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-5 F1-Score 0.9713 # 5
IoU 94.51 # 2
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-4 F1-Score 0.9685 # 7
IoU 94 # 3
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-3 F1-Score 0.9648 # 9
IoU 93.33 # 4
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-2 F1-Score 0.9552 # 12
IoU 91.62 # 5
Change detection for remote sensing images CDD Dataset (season-varying) SiamixFormer-1 F1-Score 0.9278 # 16
IoU 87 # 6
Building change detection for remote sensing images LEVIR-CD SiamixFormer-1 F1 89.29 # 23
IoU 82.03 # 12
Building change detection for remote sensing images LEVIR-CD SiamixFormer-0 F1 89.47 # 21
IoU 82.29 # 11
Building change detection for remote sensing images LEVIR-CD SiamixFormer-3 F1 90.54 # 14
IoU 83.84 # 9
Building change detection for remote sensing images LEVIR-CD SiamixFormer-2 F1 90.57 # 13
IoU 83.88 # 8
Building change detection for remote sensing images LEVIR-CD SiamixFormer-4 F1 90.70 # 12
IoU 84.05 # 6
Building change detection for remote sensing images LEVIR-CD SiamixFormer-5 F1 91.58 # 7
IoU 85.38 # 3
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-0 F1 95.53 # 6
IoU 91.43 # 6
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-5 F1 96.69 # 1
IoU 93.58 # 1
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-4 F1 96.59 # 2
IoU 93.4 # 2
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-3 F1 96.32 # 3
IoU 92.9 # 3
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-2 F1 96.22 # 4
IoU 92.7 # 4
Extracting Buildings In Remote Sensing Images WHU Building Dataset SiamixFormer-1 F1 96.01 # 5
IoU 92.31 # 5
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-0 IoU 76.43 # 6
F1 86.46 # 7
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-3 IoU 79.06 # 3
F1 88.3 # 3
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-4 IoU 79.14 # 2
F1 88.35 # 2
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-5 IoU 79.26 # 1
F1 88.43 # 1
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-2 IoU 78.66 # 4
F1 88.05 # 4
Extracting Buildings In Remote Sensing Images xBD SiamixFormer-1 IoU 77.35 # 5
F1 87.23 # 5

Methods