Swin2-MoSE: A New Single Image Super-Resolution Model for Remote Sensing

29 Apr 2024  ·  Leonardo Rossi, Vittorio Bernuzzi, Tomaso Fontanini, Massimo Bertozzi, Andrea Prati ·

Due to the limitations of current optical and sensor technologies and the high cost of updating them, the spectral and spatial resolution of satellites may not always meet desired requirements. For these reasons, Remote-Sensing Single-Image Super-Resolution (RS-SISR) techniques have gained significant interest. In this paper, we propose Swin2-MoSE model, an enhanced version of Swin2SR. Our model introduces MoE-SM, an enhanced Mixture-of-Experts (MoE) to replace the Feed-Forward inside all Transformer block. MoE-SM is designed with Smart-Merger, and new layer for merging the output of individual experts, and with a new way to split the work between experts, defining a new per-example strategy instead of the commonly used per-token one. Furthermore, we analyze how positional encodings interact with each other, demonstrating that per-channel bias and per-head bias can positively cooperate. Finally, we propose to use a combination of Normalized-Cross-Correlation (NCC) and Structural Similarity Index Measure (SSIM) losses, to avoid typical MSE loss limitations. Experimental results demonstrate that Swin2-MoSE outperforms SOTA by up to 0.377 ~ 0.958 dB (PSNR) on task of 2x, 3x and 4x resolution-upscaling (Sen2Venus and OLI2MSI datasets). We show the efficacy of Swin2-MoSE, applying it to a semantic segmentation task (SeasoNet dataset). Code and pretrained are available on https://github.com/IMPLabUniPr/swin2-mose/tree/official_code

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multispectral Image Super-resolution OLI2MSI - 3x upscaling Swin2SR-MoSE SSIM 0.9912 # 1
PSNR 45.9194 # 1
Multispectral Image Super-resolution Sen2venus - 2x upscaling Swin2SR-MoSE SSIM 0.9948 # 1
PSNR 49.4784 # 1
Multispectral Image Super-resolution Sen2venus - 4x upscaling Swin2SR-MoSE SSIM 0.9849 # 1
PSNR 45.9272 # 1