Restormer: Efficient Transformer for High-Resolution Image Restoration

Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shortcomings of CNNs (i.e., limited receptive field and inadaptability to input content), its computational complexity grows quadratically with the spatial resolution, therefore making it infeasible to apply to most image restoration tasks involving high-resolution images. In this work, we propose an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks, including image deraining, single-image motion deblurring, defocus deblurring (single-image and dual-pixel data), and image denoising (Gaussian grayscale/color denoising, and real image denoising). The source code and pre-trained models are available at https://github.com/swz30/Restormer.

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Grayscale Image Denoising BSD68 sigma15 Restormer PSNR 31.96 # 3
Image Denoising DND Restormer PSNR (sRGB) 40.03 # 1
SSIM (sRGB) 0.956 # 1
Image Defocus Deblurring DPD Restormer Combined PSNR 26.66 # 2
Combined SSIM 0.833 # 4
Deblurring GoPro Restormer PSNR 32.92 # 16
SSIM 0.961 # 15
Image Deblurring GoPro Restormer PSNR 32.92 # 9
SSIM 0.961 # 9
Deblurring HIDE (trained on GOPRO) Restormer PSNR (sRGB) 31.22 # 4
SSIM (sRGB) 0.942 # 5
Color Image Denoising Kodak24 sigma50 Restormer PSNR 30.01 # 1
Deblurring MSU BASED Restormer local SSIM 0.94217 # 11
PSNR 31.12341 # 8
VMAF 65.25911 # 11
LPIPS 0.08251 # 6
ERQAv2.0 0.73875 # 11
Subjective 0.1231 # 10
Deblurring MSU BASED Restormer SSIM 0.94632 # 3
PSNR 31.76111 # 1
VMAF 66.3964 # 10
LPIPS 0.08239 # 4
ERQAv2.0 0.74776 # 3
Subjective 0.1175 # 11
Single Image Deraining Rain100H Restormer PSNR 31.46 # 1
SSIM 0.904 # 2
Single Image Deraining Rain100L Restormer PSNR 38.99 # 3
SSIM 0.978 # 4
Deblurring RealBlur-J (trained on GoPro) Restormer PSNR (sRGB) 28.96 # 3
SSIM (sRGB) 0.879 # 3
Deblurring RealBlur-R (trained on GoPro) Restormer PSNR (sRGB) 36.19 # 2
SSIM (sRGB) 0.957 # 1
Deblurring RSBlur Restormer Average PSNR 33.69 # 2
Image Denoising SIDD Restormer PSNR (sRGB) 40.02 # 3
SSIM (sRGB) 0.960 # 3
Single Image Deraining Test100 Restormer PSNR 32.00 # 1
SSIM 0.923 # 1
Single Image Deraining Test1200 Restormer PSNR 33.19 # 1
SSIM 0.926 # 1
Single Image Deraining Test2800 Restormer PSNR 34.18 # 1
SSIM 0.944 # 1
Grayscale Image Denoising Urban100 sigma15 Restormer PSNR 33.79 # 1
Color Image Denoising Urban100 sigma25 Restormer PSNR 32.96 # 1
Grayscale Image Denoising Urban100 sigma25 Restormer PSNR 31.46 # 2
Color Image Denoising Urban100 sigma50 Restormer PSNR 30.02 # 2
Grayscale Image Denoising Urban100 sigma50 Restormer PSNR 28.29 # 2

Methods