Multi-Stage Progressive Image Restoration

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Spectral Reconstruction ARAD-1K MPRNet PSNR 33.50 # 3
MRAE 0.1817 # 3
RMSE 0.0270 # 3
Image Denoising DND MPRNet PSNR (sRGB) 39.80 # 8
SSIM (sRGB) 0.954 # 9
Deblurring GoPro MPRNet PSNR 32.66 # 32
SSIM 0.959 # 29
Image Deblurring GoPro MPRNet PSNR 32.66 # 30
SSIM 0.959 # 27
Params (M) 20.1 # 10
Deblurring HIDE (trained on GOPRO) MPRNet PSNR (sRGB) 30.96 # 16
SSIM (sRGB) 0.939 # 17
Params (M) 20.1 # 6
Single Image Deraining Rain100H MPRNet PSNR 30.41 # 8
SSIM 0.89 # 10
Single Image Deraining Rain100L MPRNet PSNR 36.40 # 12
SSIM 0.965 # 13
Deblurring RealBlur-J MPRNet SSIM (sRGB) 0.922 # 14
PSNR (sRGB) 31.76 # 15
Params(M) 20.1 # 7
Deblurring RealBlur-J (trained on GoPro) MPRNet PSNR (sRGB) 28.70 # 10
SSIM (sRGB) 0.873 # 10
Deblurring RealBlur-R MPRNet PSNR (sRGB) 39.31 # 14
SSIM (sRGB) 0.972 # 11
Deblurring RealBlur-R (trained on GoPro) MPRNet PSNR (sRGB) 35.99 # 8
SSIM (sRGB) 0.952 # 9
Deblurring RSBlur MPRNet Average PSNR 33.61 # 5
Image Denoising SIDD MPRNet PSNR (sRGB) 39.71 # 12
SSIM (sRGB) 0.958 # 12
Single Image Deraining Test100 MPRNet PSNR 30.27 # 6
SSIM 0.897 # 6
Single Image Deraining Test1200 MPRNet PSNR 32.91 # 7
SSIM 0.916 # 8
Single Image Deraining Test2800 MPRNet PSNR 33.64 # 5
SSIM 0.938 # 4

Methods