Deep Bayesian Video Frame Interpolation
Abstract. We present deep Bayesian video frame interpolation, a novel approach for upsampling a low frame-rate video temporally to its higher frame-rate counterpart. Our approach learns posterior distributions of optical flows and frames to be interpolated, which is optimized via learned gradient descent for fast convergence. Each learned step is a lightweight network manipulating gradients of the log-likelihood of estimated frames and flows. Such gradients, parameterized either explicitly or implicitly, model the fidelity of current estimations when matching real image and flow distributions to explain the input observations. With this approach we show new records on 8 of 10 benchmarks, using an architecture with half the parameters of the state-of-the-art model.
PDF AbstractCode
Datasets
UCF101
DAVIS
GoPro
X4K1000FPS
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Video Frame Interpolation | DAVIS | DBVI | PSNR | 28.61 | # 1 | |
| SSIM | 0.905 | # 1 | ||||
| Video Frame Interpolation | GoPro | DBVI | PSNR | 31.73 | # 1 | |
| SSIM | 0.947 | # 1 | ||||
| Video Frame Interpolation | SNU-FILM (easy) | DBVI | PSNR | 40.46 | # 2 | |
| SSIM | 0.991 | # 2 | ||||
| Video Frame Interpolation | SNU-FILM (extreme) | DBVI | PSNR | 25.90 | # 1 | |
| SSIM | 0.876 | # 1 | ||||
| Video Frame Interpolation | SNU-FILM (hard) | DBVI | PSNR | 31.68 | # 2 | |
| SSIM | 0.953 | # 1 | ||||
| Video Frame Interpolation | SNU-FILM (medium) | DBVI | PSNR | 36.95 | # 2 | |
| SSIM | 0.985 | # 1 | ||||
| Video Frame Interpolation | X4K1000FPS | DBVI | PSNR | 32.89 | # 1 | |
| SSIM | 0.939 | # 1 |
