Accelerating the Super-Resolution Convolutional Neural Network

1 Aug 2016  ·  Chao Dong, Chen Change Loy, Xiaoou Tang ·

As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) has demonstrated superior performance to the previous hand-crafted models either in speed and restoration quality. However, the high computational cost still hinders it from practical usage that demands real-time performance (24 fps). In this paper, we aim at accelerating the current SRCNN, and propose a compact hourglass-shape CNN structure for faster and better SR. We re-design the SRCNN structure mainly in three aspects. First, we introduce a deconvolution layer at the end of the network, then the mapping is learned directly from the original low-resolution image (without interpolation) to the high-resolution one. Second, we reformulate the mapping layer by shrinking the input feature dimension before mapping and expanding back afterwards. Third, we adopt smaller filter sizes but more mapping layers. The proposed model achieves a speed up of more than 40 times with even superior restoration quality. Further, we present the parameter settings that can achieve real-time performance on a generic CPU while still maintaining good performance. A corresponding transfer strategy is also proposed for fast training and testing across different upscaling factors.

PDF Abstract


Introduced in the Paper:


Used in the Paper:

FFHQ BSD Set14 Set5 BSD100
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution BSD100 - 2x upscaling FSRCNN [[Dong et al.2016]] PSNR 31.53 # 21
Image Super-Resolution FFHQ 1024 x 1024 - 4x upscaling FSRCNN FID 23.97 # 6
MS-SSIM 0.951 # 3
PSNR 24.71 # 7
SSIM 0.804 # 6

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Image Super-Resolution FFHQ 256 x 256 - 4x upscaling FSRCNN FID 139.78 # 6
MS-SSIM 0.930 # 3
PSNR 22.45 # 7
SSIM 0.709 # 3