Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

22 Jul 2021  ·  Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan ·

Though many attempts have been made in blind super-resolution to restore low-resolution images with unknown and complex degradations, they are still far from addressing general real-world degraded images. In this work, we extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN), which is trained with pure synthetic data. Specifically, a high-order degradation modeling process is introduced to better simulate complex real-world degradations. We also consider the common ringing and overshoot artifacts in the synthesis process. In addition, we employ a U-Net discriminator with spectral normalization to increase discriminator capability and stabilize the training dynamics. Extensive comparisons have shown its superior visual performance than prior works on various real datasets. We also provide efficient implementations to synthesize training pairs on the fly.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Video Super-Resolution MSU Super-Resolution for Video Compression Real-ESRGAN + x264 BSQ-rate over ERQA 5.58 # 26
Compare
BSQ-rate over Subjective Score 0.335 # 4
Compare
BSQ-rate over VMAF 0.698 # 5
Compare
BSQ-rate over PSNR 7.874 # 42
Compare
BSQ-rate over MS-SSIM 0.881 # 16
Compare
BSQ-rate over LPIPS 0.733 # 4
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression Real-ESRGAN + x265 BSQ-rate over ERQA 6.328 # 28
Compare
BSQ-rate over Subjective Score 0.64 # 10
Compare
BSQ-rate over VMAF 1.464 # 30
Compare
BSQ-rate over PSNR 8.113 # 44
Compare
BSQ-rate over MS-SSIM 5.393 # 61
Compare
BSQ-rate over LPIPS 12.689 # 68
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression Real-ESRGAN + uavs3e BSQ-rate over ERQA 7.225 # 37
Compare
BSQ-rate over Subjective Score 1.417 # 23
Compare
BSQ-rate over VMAF 2.122 # 48
Compare
BSQ-rate over PSNR 15.144 # 73
Compare
BSQ-rate over MS-SSIM 4.612 # 56
Compare
BSQ-rate over LPIPS 2.633 # 28
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression Real-ESRGAN + aomenc BSQ-rate over ERQA 11.584 # 49
Compare
BSQ-rate over Subjective Score 1.398 # 22
Compare
BSQ-rate over VMAF 2.712 # 52
Compare
BSQ-rate over PSNR 15.144 # 73
Compare
BSQ-rate over MS-SSIM 6.857 # 74
Compare
BSQ-rate over LPIPS 11.957 # 67
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression Real-ESRGAN + vvenc BSQ-rate over ERQA 6.712 # 32
Compare
BSQ-rate over VMAF 3.8 # 58
Compare
BSQ-rate over PSNR 14.561 # 71
Compare
BSQ-rate over MS-SSIM 5.95 # 68
Compare
BSQ-rate over LPIPS 12.744 # 69
Compare
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration Real-ESRnet Subjective score 3.697 # 26
Compare
ERQAv1.0 0.598 # 27
Compare
QRCRv1.0 0 # 21
Compare
SSIM 0.824 # 21
Compare
PSNR 27.195 # 16
Compare
FPS 1.019 # 13
Compare
1 - LPIPS 0.871 # 17
Compare
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration Real-ESRGAN Subjective score 5.392 # 13
Compare
ERQAv1.0 0.663 # 16
Compare
QRCRv1.0 0 # 21
Compare
SSIM 0.774 # 29
Compare
PSNR 24.441 # 29
Compare
FPS 1.01 # 14
Compare
1 - LPIPS 0.895 # 11
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrgan-F PSNR 28.82 # 29
Compare
LPIPS 0.185 # 5
Compare
SSIM 0.850 # 13
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrnet-F PSNR 30.01 # 22
Compare
LPIPS 0.280 # 20
Compare
SSIM 0.868 # 21
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrgan-V PSNR 25.52 # 48
Compare
LPIPS 0.333 # 27
Compare
SSIM 0.795 # 1
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrnet PSNR 30.52 # 17
Compare
LPIPS 0.296 # 23
Compare
SSIM 0.878 # 26
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrgan PSNR 29.14 # 26
Compare
LPIPS 0.181 # 3
Compare
SSIM 0.855 # 18
Compare
Video Super-Resolution MSU Video Upscalers: Quality Enhancement RealEsrgan-A PSNR 28.71 # 30
Compare
LPIPS 0.244 # 16
Compare
SSIM 0.830 # 4
Compare

Methods
Add Remove

Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy