MemNet: A Persistent Memory Network for Image Restoration

ICCV 2017  ·  Ying Tai, Jian Yang, Xiaoming Liu, Chunyan Xu ·

Recently, very deep convolutional neural networks (CNNs) have been attracting considerable attention in image restoration. However, as the depth grows, the long-term dependency problem is rarely realized for these very deep models, which results in the prior states/layers having little influence on the subsequent ones... Motivated by the fact that human thoughts have persistency, we propose a very deep persistent memory network (MemNet) that introduces a memory block, consisting of a recursive unit and a gate unit, to explicitly mine persistent memory through an adaptive learning process. The recursive unit learns multi-level representations of the current state under different receptive fields. The representations and the outputs from the previous memory blocks are concatenated and sent to the gate unit, which adaptively controls how much of the previous states should be reserved, and decides how much of the current state should be stored. We apply MemNet to three image restoration tasks, i.e., image denosing, super-resolution and JPEG deblocking. Comprehensive experiments demonstrate the necessity of the MemNet and its unanimous superiority on all three tasks over the state of the arts. Code is available at https://github.com/tyshiwo/MemNet. read more

PDF Abstract ICCV 2017 PDF ICCV 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution BSD100 - 4x upscaling MemNet PSNR 27.40 # 31
SSIM 0.7281 # 34
Color Image Denoising CBSD68 sigma50 MemNet PSNR 26.33 # 10
JPEG Artifact Correction Classic5 (Quality 10 Grayscale) MemNet PSNR 29.69 # 4
JPEG Artifact Correction LIVE1 (Quality 10 Color) MemNet PSNR 27.33 # 5
PSNR-B 27.34 # 5
SSIM 0.810 # 3
JPEG Artifact Correction Live1 (Quality 10 Grayscale) MemNet PSNR 29.45 # 5
PSNR-B 29.39 # 2
SSIM 0.8327 # 4
JPEG Artifact Correction LIVE1 (Quality 20 Color) MemNet PSNR 29.76 # 5
PSNR-B 29.75 # 4
SSIM 0.877 # 4
JPEG Artifact Correction LIVE1 (Quality 20 Grayscale) MemNet PSNR 31.83 # 5
PSNR-B 31.74 # 4
SSIM 0.8970 # 5
Image Super-Resolution Manga109 - 4x upscaling MemNet PSNR 29.42 # 20
SSIM 0.8942 # 19
Image Super-Resolution Set14 - 4x upscaling MemNet PSNR 28.26 # 32
SSIM 0.7723 # 34
Image Super-Resolution Set5 - 4x upscaling MemNet PSNR 31.74 # 33
SSIM 0.8893 # 34
Image Super-Resolution Urban100 - 4x upscaling MemNet PSNR 25.50 # 27
SSIM 0.7630 # 30

Methods