Learning to Deblur using Light Field Generated and Real Defocus Images

CVPR 2022  ยท  Lingyan Ruan, Bin Chen, Jizhou Li, Miuling Lam ยท

Defocus deblurring is a challenging task due to the spatially varying nature of defocus blur. While deep learning approach shows great promise in solving image restoration problems, defocus deblurring demands accurate training data that consists of all-in-focus and defocus image pairs, which is difficult to collect. Naive two-shot capturing cannot achieve pixel-wise correspondence between the defocused and all-in-focus image pairs. Synthetic aperture of light fields is suggested to be a more reliable way to generate accurate image pairs. However, the defocus blur generated from light field data is different from that of the images captured with a traditional digital camera. In this paper, we propose a novel deep defocus deblurring network that leverages the strength and overcomes the shortcoming of light fields. We first train the network on a light field-generated dataset for its highly accurate image correspondence. Then, we fine-tune the network using feature loss on another dataset collected by the two-shot method to alleviate the differences between the defocus blur exists in the two domains. This strategy is proved to be highly effective and able to achieve the state-of-the-art performance both quantitatively and qualitatively on multiple test sets. Extensive ablation studies have been conducted to analyze the effect of each network module to the final performance.

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract

Results from the Paper


 Ranked #1 on Image Defocus Deblurring on RealDOF (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Defocus Deblurring DPD DRBNet Combined PSNR 25.47 # 4
Combined SSIM 0.787 # 6
LPIPS 0.246 # 7
Image Defocus Deblurring DPD (Dual-view) DRBNet PSNR 26.33 # 7
SSIM 0.811 # 10
LPIPS 0.154 # 3
Image Defocus Deblurring RealDOF DRBNet PSNR 25.75 # 1
SSIM 0.771 # 1
LPIPS 0.257 # 1

Methods


No methods listed for this paper. Add relevant methods here