Multi-Scale Progressive Fusion Network for Single Image Deraining

Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions, which makes it possible to exploit such complementary information for rain streak representation. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features in a unified framework, termed multi-scale progressive fusion network (MSPFN) for single image rain streak removal. For similar rain streaks at different positions, we employ recurrent calculation to capture the global texture, thus allowing to explore the complementary and redundant information at the spatial dimension to characterize target rain streaks. Besides, we construct multi-scale pyramid structure, and further introduce the attention mechanism to guide the fine fusion of this correlated information from different scales. This multi-scale progressive fusion strategy not only promotes the cooperative representation, but also boosts the end-to-end training. Our proposed method is extensively evaluated on several benchmark datasets and achieves state-of-the-art results. Moreover, we conduct experiments on joint deraining, detection, and segmentation tasks, and inspire a new research direction of vision task-driven image deraining. The source code is available at \url{https://github.com/kuihua/MSPFN}.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract

Datasets


Introduced in the Paper:

Synthetic Rain Datasets

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Single Image Deraining Rain100H MSPFN PSNR 28.66 # 11
SSIM 0.86 # 11
Single Image Deraining Rain100L MSPFN PSNR 32.40 # 13
SSIM 0.933 # 14
Single Image Deraining Test100 MSPFN PSNR 27.50 # 7
SSIM 0.876 # 7
Single Image Deraining Test1200 MSPFN PSNR 32.39 # 8
SSIM 0.916 # 8
Single Image Deraining Test2800 MSPFN PSNR 32.82 # 6
SSIM 0.930 # 5

Methods