Learning Parallax Attention for Stereo Image Super-Resolution

Stereo image pairs can be used to improve the performance of super-resolution (SR) since additional information is provided from a second viewpoint. However, it is challenging to incorporate this information for SR since disparities between stereo images vary significantly. In this paper, we propose a parallax-attention stereo superresolution network (PASSRnet) to integrate the information from a stereo image pair for SR. Specifically, we introduce a parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations. We also propose a new and the largest dataset for stereo image SR (namely, Flickr1024). Extensive experiments demonstrate that the parallax-attention mechanism can capture correspondence between stereo images to improve SR performance with a small computational and memory cost. Comparative results show that our PASSRnet achieves the state-of-the-art performance on the Middlebury, KITTI 2012 and KITTI 2015 datasets.

PDF Abstract CVPR 2019 PDF CVPR 2019 Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Stereo Image Super-Resolution Flickr1024 - 2x upscaling PASSRnet PSNR 28.38 # 6
Stereo Image Super-Resolution Flickr1024 - 4x upscaling PASSRnet PSNR 23.31 # 6
Stereo Image Super-Resolution KITTI2012 - 2x upscaling PASSRnet PSNR 30.81 # 3
Image Super-Resolution KITTI 2012 - 2x upscaling PASSRnet PSNR 30.65 # 1
Image Super-Resolution KITTI 2012 - 4x upscaling PASSRnet PSNR 26.26 # 1
Stereo Image Super-Resolution KITTI2012 - 4x upscaling PASSRnet PSNR 26.34 # 6
Image Super-Resolution KITTI 2015 - 2x upscaling PASSRnet PSNR 29.78 # 1
Stereo Image Super-Resolution KITTI2015 - 2x upscaling PASSRnet PSNR 30.60 # 6
Image Super-Resolution KITTI 2015 - 4x upscaling PASSRnet PSNR 25.43 # 1
Stereo Image Super-Resolution KITTI2015 - 4x upscaling PASSRnet PSNR 26.08 # 6
Stereo Image Super-Resolution Middlebury - 2x upscaling PASSRnet PSNR 34.23 # 7
Image Super-Resolution Middlebury - 2x upscaling PASSRnet PSNR 34.05 # 1
Stereo Image Super-Resolution Middlebury - 4x upscaling PASSRnet PSNR 28.72 # 6
Image Super-Resolution Middlebury - 4x upscaling PASSRnet PSNR 28.63 # 1

Methods


No methods listed for this paper. Add relevant methods here