Patch Craft: Video Denoising by Deep Modeling and Patch Matching

The non-local self-similarity property of natural images has been exploited extensively for solving various image processing problems. When it comes to video sequences, harnessing this force is even more beneficial due to the temporal redundancy. In the context of image and video denoising, many classically-oriented algorithms employ self-similarity, splitting the data into overlapping patches, gathering groups of similar ones and processing these together somehow. With the emergence of convolutional neural networks (CNN), the patch-based framework has been abandoned. Most CNN denoisers operate on the whole image, leveraging non-local relations only implicitly by using a large receptive field. This work proposes a novel approach for leveraging self-similarity in the context of video denoising, while still relying on a regular convolutional architecture. We introduce a concept of patch-craft frames - artificial frames that are similar to the real ones, built by tiling matched patches. Our algorithm augments video sequences with patch-craft frames and feeds them to a CNN. We demonstrate the substantial boost in denoising performance obtained with the proposed approach.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Color Image Denoising CBSD68 sigma15 PaCNet PSNR 33.95 # 5
Color Image Denoising CBSD68 sigma25 PaCNet PSNR 31.22 # 4
Color Image Denoising CBSD68 sigma50 PaCNet PSNR 27.93 # 8
Video Denoising DAVIS sigma10 PaCNet PSNR 39.97 # 4
Video Denoising DAVIS sigma20 PaCNet PSNR 36.82 # 4
Video Denoising DAVIS sigma30 PaCNet PSNR 34.79 # 4
Video Denoising DAVIS sigma40 PaCNet PSNR 33.34 # 5
Video Denoising DAVIS sigma50 PaCNet PSNR 32.2 # 5
Video Denoising Set8 sigma10 PaCNet PSNR 37.06 # 4
Video Denoising Set8 sigma20 PaCNet PSNR 33.94 # 4
Video Denoising Set8 sigma30 PaCNet PSNR 32.05 # 4
Video Denoising Set8 sigma40 PaCNet PSNR 30.7 # 6
Video Denoising Set8 sigma50 PaCNet PSNR 29.66 # 6


No methods listed for this paper. Add relevant methods here