Multi-level Wavelet-CNN for Image Restoration

18 May 2018  ·  Pengju Liu, Hongzhi Zhang, Kai Zhang, Liang Lin, WangMeng Zuo ·

The tradeoff between receptive field size and efficiency is a crucial issue in low level vision. Plain convolutional networks (CNNs) generally enlarge the receptive field at the expense of computational cost... Recently, dilated filtering has been adopted to address this issue. But it suffers from gridding effect, and the resulting receptive field is only a sparse sampling of input image with checkerboard patterns. In this paper, we present a novel multi-level wavelet CNN (MWCNN) model for better tradeoff between receptive field size and computational efficiency. With the modified U-Net architecture, wavelet transform is introduced to reduce the size of feature maps in the contracting subnetwork. Furthermore, another convolutional layer is further used to decrease the channels of feature maps. In the expanding subnetwork, inverse wavelet transform is then deployed to reconstruct the high resolution feature maps. Our MWCNN can also be explained as the generalization of dilated filtering and subsampling, and can be applied to many image restoration tasks. The experimental results clearly show the effectiveness of MWCNN for image denoising, single image super-resolution, and JPEG image artifacts removal. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution BSD100 - 2x upscaling MWCNN PSNR 32.23 # 7
Image Super-Resolution BSD100 - 3x upscaling MWCNN PSNR 29.12 # 4
Image Super-Resolution BSD100 - 4x upscaling MWCNN PSNR 27.62 # 18
SSIM 0.7355 # 23
Grayscale Image Denoising BSD68 sigma15 MWCNN PSNR 31.86 # 2
Grayscale Image Denoising BSD68 sigma25 MWCNN PSNR 29.41 # 1
Grayscale Image Denoising BSD68 sigma50 MWCNN PSNR 26.53 # 1
JPEG Artifact Correction Classic5 (Quality 10 Grayscale) MWCNN PSNR 30.01 # 2
JPEG Artifact Correction Classic5 (Quality 20 Grayscale) MWCNN PSNR 32.16 # 2
JPEG Artifact Correction Classic5 (Quality 30 Grayscale) MWCNN PSNR 33.43 # 2
JPEG Artifact Correction Classic5 (Quality 40 Grayscale) MWCNN PSNR 34.27 # 2
JPEG Artifact Correction ICB (Quality 10 Color) MWCNN PSNR 30.76 # 4
PSNR-B 31.21 # 4
SSIM 0.779 # 4
JPEG Artifact Correction ICB (Quality 10 Grayscale) MWCNN PSNR 34.12 # 3
PSNR-B 34.06 # 3
SSIM 0.884 # 2
JPEG Artifact Correction ICB (Quality 20 Color) MWCNN PSNR 32.79 # 3
PSNR-B 33.32 # 3
SSIM 0.812 # 4
JPEG Artifact Correction ICB (Quality 20 Grayscale) MWCNN PSNR 36.56 # 2
PSNR-B 36.44 # 2
SSIM 0.902 # 4
JPEG Artifact Correction ICB (Quality 30 Color) MWCNN PSNR 34.11 # 2
PSNR-B 34.69 # 2
SSIM 0.845 # 2
JPEG Artifact Correction LIVE1 (Quality 10 Color) MWCNN PSNR 27.45 # 3
PSNR-B 27.44 # 2
SSIM 0.808 # 5
JPEG Artifact Correction Live1 (Quality 10 Grayscale) MWCNN PSNR 29.69 # 3
PSNR-B 29.39 # 2
SSIM 0.8357 # 3
JPEG Artifact Correction LIVE1 (Quality 20 Color) MWCNN PSNR 29.80 # 4
PSNR-B 29.78 # 3
SSIM 0.877 # 4
JPEG Artifact Correction LIVE1 (Quality 20 Grayscale) MWCNN PSNR 32.04 # 3
PSNR-B 31.83 # 2
SSIM 0.8989 # 3
JPEG Artifact Correction LIVE1 (Quality 30 Grayscale) MWCNN PSNR 33.45 # 2
JPEG Artifact Correction LIVE1 (Quality 40 Grayscale) MWCNN PSNR 34.45 # 2
Grayscale Image Denoising Set12 sigma15 MWCNN PSNR 33.15 # 2
Grayscale Image Denoising Set12 sigma25 MWCNN PSNR 30.79 # 1
Grayscale Image Denoising Set12 sigma50 MWCNN PSNR 27.74 # 1
Image Super-Resolution Set14 - 2x upscaling MWCNN PSNR 33.7 # 8
Image Super-Resolution Set14 - 3x upscaling MWCNN PSNR 30.16 # 4
Image Super-Resolution Set14 - 4x upscaling MWCNN PSNR 28.41 # 27
SSIM 0.7816 # 25
Image Super-Resolution Set5 - 2x upscaling MWCNN PSNR 37.91 # 11
Image Super-Resolution Set5 - 3x upscaling MWCNN PSNR 34.17 # 7
Image Super-Resolution Set5 - 4x upscaling MWCNN PSNR 32.12 # 25
SSIM 0.8941 # 23
Image Super-Resolution Urban100 - 2x upscaling MWCNN PSNR 32.3 # 9
Image Super-Resolution Urban100 - 3x upscaling MWCNN PSNR 28.13 # 5
Image Super-Resolution Urban100 - 4x upscaling MWCNN PSNR 26.27 # 18
SSIM 0.7890 # 19
Grayscale Image Denoising Urban100 sigma15 MWCNN PSNR 33.17 # 3
Grayscale Image Denoising Urban100 sigma25 MWCNN PSNR 30.66 # 3
Grayscale Image Denoising Urban100 sigma50 MWCNN PSNR 27.42 # 3

Methods