BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment

18 Apr 2022  ·  Ziwei Luo, Youwei Li, Shen Cheng, Lei Yu, Qi Wu, Zhihong Wen, Haoqiang Fan, Jian Sun, Shuaicheng Liu ·

This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To overcome the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Transformer Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Burst Image Super-Resolution BurstSR BSRT-Large PSNR 48.57 # 1
SSIM 0.986 # 1
LPIPS 0.021 # 5
Burst Image Super-Resolution BurstSR BSRT-Small PSNR 48.48 # 3
SSIM 0.985 # 3
LPIPS 0.021 # 5
Burst Image Super-Resolution SyntheticBurst BSRT-Large PSNR 43.62 # 1
SSIM 0.975 # 1
LPIPS 0.025 # 5
Burst Image Super-Resolution SyntheticBurst BSRT-Small PSNR 42.72 # 3
SSIM 0.971 # 3
LPIPS 0.031 # 3