BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond

Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. In this study, we wish to untangle the knots and reconsider some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, we show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. We conduct systematic analysis to explain how such gain can be obtained and discuss the pitfalls. We further show the extensibility of BasicVSR by presenting an information-refill mechanism and a coupled propagation scheme to facilitate information aggregation. The BasicVSR and its extension, IconVSR, can serve as strong baselines for future VSR approaches.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Video Super-Resolution MSU Super-Resolution for Video Compression BasicVSR + vvenc BSQ-rate over ERQA 18.333 # 74
BSQ-rate over Subjective Score 2.659 # 35
BSQ-rate over VMAF 0.676 # 3
BSQ-rate over PSNR 5.781 # 26
BSQ-rate over MS-SSIM 0.919 # 20
BSQ-rate over LPIPS 11.561 # 64
Video Super-Resolution MSU Super-Resolution for Video Compression BasicVSR + uavs3e BSQ-rate over ERQA 8.251 # 39
BSQ-rate over Subjective Score 2.724 # 39
BSQ-rate over VMAF 1.523 # 34
BSQ-rate over PSNR 6.833 # 39
BSQ-rate over MS-SSIM 2.261 # 39
BSQ-rate over LPIPS 4.383 # 36
Video Super-Resolution MSU Super-Resolution for Video Compression BasicVSR + x265 BSQ-rate over ERQA 8.921 # 41
BSQ-rate over Subjective Score 2.238 # 30
BSQ-rate over VMAF 1.272 # 22
BSQ-rate over PSNR 1.906 # 15
BSQ-rate over MS-SSIM 1.48 # 30
BSQ-rate over LPIPS 13.198 # 72
Video Super-Resolution MSU Super-Resolution for Video Compression BasicVSR + aomenc BSQ-rate over ERQA 14.568 # 64
BSQ-rate over Subjective Score 2.673 # 36
BSQ-rate over VMAF 1.857 # 43
BSQ-rate over PSNR 11.428 # 62
BSQ-rate over MS-SSIM 4.128 # 50
BSQ-rate over LPIPS 4.938 # 45
Video Super-Resolution MSU Super-Resolution for Video Compression BasicVSR + x264 BSQ-rate over ERQA 1.659 # 12
BSQ-rate over Subjective Score 1.49 # 24
BSQ-rate over VMAF 0.714 # 9
BSQ-rate over PSNR 1.212 # 10
BSQ-rate over MS-SSIM 0.751 # 10
BSQ-rate over LPIPS 1.289 # 15
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration BasicVSR Subjective score 7.186 # 2
ERQAv1.0 0.75 # 2
QRCRv1.0 0.709 # 2
SSIM 0.9 # 2
PSNR 31.443 # 2
FPS 2.128 # 7
1 - LPIPS 0.934 # 2
Video Super-Resolution UDM10 - 4x upscaling IconVSR PSNR 40.03 # 5
SSIM 0.9694 # 5
Video Super-Resolution UDM10 - 4x upscaling BasicVSR PSNR 39.96 # 6
SSIM 0.9694 # 5
Video Super-Resolution Vid4 - 4x upscaling BasicVSR PSNR 27.24 # 9
SSIM 0.8251 # 9
Video Super-Resolution Vid4 - 4x upscaling IconVSR PSNR 27.39 # 6
SSIM 0.8279 # 7
Video Super-Resolution Vid4 - 4x upscaling - BD degradation IconVSR PSNR 28.04 # 7
SSIM 0.8570 # 7
Video Super-Resolution Vid4 - 4x upscaling - BD degradation BasicVSR PSNR 27.96 # 8
SSIM 0.8553 # 8

Methods