Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining

Deep convolution-based single image super-resolution (SISR) networks embrace the benefits of learning from large-scale external image resources for local recovery, yet most existing works have ignored the long-range feature-wise similarities in natural images. Some recent works have successfully leveraged this intrinsic feature correlation by exploring non-local attention modules. However, none of the current deep models have studied another inherent property of images: cross-scale feature correlation. In this paper, we propose the first Cross-Scale Non-Local (CS-NL) attention module with integration into a recurrent neural network. By combining the new CS-NL prior with local and in-scale non-local priors in a powerful recurrent fusion cell, we can find more cross-scale feature correlations within a single low-resolution (LR) image. The performance of SISR is significantly improved by exhaustively integrating all possible priors. Extensive experiments demonstrate the effectiveness of the proposed CS-NL module by setting new state-of-the-arts on multiple SISR benchmarks.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution BSD100 - 2x upscaling CSNLN PSNR 32.4 # 16
SSIM 0.9024 # 12
Image Super-Resolution BSD100 - 3x upscaling CSNLN PSNR 29.33 # 11
SSIM 0.8105 # 10
Image Super-Resolution BSD100 - 4x upscaling CSNLN PSNR 27.8 # 18
SSIM 0.7439 # 21
Image Super-Resolution Manga109 - 2x upscaling CSNLN PSNR 39.37 # 12
SSIM 0.9785 # 12
Image Super-Resolution Manga109 - 3x upscaling CSNLN PSNR 34.45 # 10
SSIM 0.9502 # 9
Image Super-Resolution Manga109 - 4x upscaling CSNLN PSNR 31.43 # 24
SSIM 0.9201 # 21
Image Super-Resolution Set14 - 2x upscaling CSNLN PSNR 34.12 # 14
SSIM 0.9223 # 12
Image Super-Resolution Set14 - 3x upscaling CSNLN PSNR 30.66 # 12
SSIM 0.8482 # 10
Image Super-Resolution Set14 - 4x upscaling CSNLN PSNR 28.95 # 29
SSIM 0.7888 # 36
Image Super-Resolution Set5 - 2x upscaling CSNLN PSNR 38.28 # 14
SSIM 0.9616 # 11
Image Super-Resolution Set5 - 3x upscaling CSNLN PSNR 34.74 # 14
SSIM 0.9300 # 11
Image Super-Resolution Urban100 - 2x upscaling CSNLN PSNR 33.25 # 13
SSIM 0.9386 # 11
Image Super-Resolution Urban100 - 3x upscaling CSNLN PSNR 29.13 # 12
SSIM 0.8712 # 8
Image Super-Resolution Urban100 - 4x upscaling CSNLN PSNR 27.22 # 15
SSIM 0.8168 # 15

Methods