Physics-Guided ISO-Dependent Sensor Noise Modeling for Extreme Low-Light Photography
Although deep neural networks have achieved astonishing performance in many vision tasks, existing learning-based methods are far inferior to the physical model-based solutions in extreme low-light sensor noise modeling. To tap the potential of learning-based sensor noise modeling, we investigate the noise formation in a typical imaging process and propose a novel physics-guided ISO-dependent sensor noise modeling approach. Specifically, we build a normalizing flow-based framework to represent the complex noise characteristics of CMOS camera sensors. Each component of the noise model is dedicated to a particular kind of noise under the guidance of physical models. Moreover, we take into consideration of the ISO dependence in the noise model, which is not completely considered by the existing learning-based methods. For training the proposed noise model, a new dataset is further collected with paired noisy-clean images, as well as flat-field and bias frames covering a wide range of ISO settings. Compared to existing methods, the proposed noise model benefits from the flexible structure and accurate modeling capabilities, which can help achieve better denoising performance in extreme low-light scenes. The source code and collected dataset will be publicly available.
PDF AbstractTasks
SIDD
SID
ELD
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Image Denoising | ELD SonyA7S2 x100 | LLD* | PSNR (Raw) | 46.74 | # 1 | |
| SSIM (Raw) | 0.986 | # 1 | ||||
| Image Denoising | ELD SonyA7S2 x100 | LLD | PSNR (Raw) | 45.61 | # 4 | |
| SSIM (Raw) | 0.977 | # 4 | ||||
| Image Denoising | ELD SonyA7S2 x200 | LLD* | PSNR (Raw) | 44.95 | # 1 | |
| SSIM (Raw) | 0.977 | # 1 | ||||
| Image Denoising | ELD SonyA7S2 x200 | LLD | PSNR (Raw) | 43.84 | # 4 | |
| SSIM (Raw) | 0.959 | # 5 | ||||
| Image Denoising | SID SonyA7S2 x250 | LLD | PSNR (Raw) | 39.76 | # 4 | |
| SSIM (Raw) | 0.933 | # 5 | ||||
| Image Denoising | SID SonyA7S2 x250 | LLD* | PSNR (Raw) | 41.02 | # 1 | |
| SSIM (Raw) | 0.948 | # 1 | ||||
| Image Denoising | SID x100 | LLD* | PSNR (Raw) | 43.36 | # 1 | |
| SSIM | 0.961 | # 2 | ||||
| Image Denoising | SID x100 | LLD | PSNR (Raw) | 42.10 | # 4 | |
| SSIM | 0.955 | # 4 | ||||
| Image Denoising | SID x300 | LLD* | PSNR (Raw) | 37.80 | # 1 | |
| SSIM | 0.935 | # 1 | ||||
| Image Denoising | SID x300 | LLD | PSNR (Raw) | 36.76 | # 5 | |
| SSIM | 0.912 | # 5 |
