Depth estimation from 4D light field videos

5 Dec 2020  ·  Takahiro Kinoshita, Satoshi Ono ·

Depth (disparity) estimation from 4D Light Field (LF) images has been a research topic for the last couple of years. Most studies have focused on depth estimation from static 4D LF images while not considering temporal information, i.e., LF videos. This paper proposes an end-to-end neural network architecture for depth estimation from 4D LF videos. This study also constructs a medium-scale synthetic 4D LF video dataset that can be used for training deep learning-based methods. Experimental results using synthetic and real-world 4D LF videos show that temporal information contributes to the improvement of depth estimation accuracy in noisy regions. Dataset and code is available at: https://mediaeng-lfv.github.io/LFV_Disparity_Estimation

PDF Abstract

Datasets


Introduced in the Paper:

Sintel 4D LFV
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Disparity Estimation Sintel 4D LFV - ambushfight5 Two-stream CNN+CLSTM MSE*100 21.67 # 1
BadPix(0.07) 8.3404 # 1
BadPix(0.03) 22.8762 # 1
BadPix(0.01) 62.0493 # 1
Disparity Estimation Sintel 4D LFV - bamboo3 Two-stream CNN+CLSTM MSE*100 21.59 # 1
BadPix(0.07) 8.9475 # 1
BadPix(0.03) 21.8162 # 1
BadPix(0.01) 53.2985 # 1
Disparity Estimation Sintel 4D LFV - shaman2 Two-stream CNN+CLSTM MSE*100 2.4421 # 1
BadPix(0.07) 32.7585 # 1
BadPix(0.03) 50.6706 # 1
BadPix(0.01) 74.7733 # 1
Disparity Estimation Sintel 4D LFV - thebigfight2 Two-stream CNN+CLSTM MSE*100 3.67 # 1
BadPix(0.05) 1.0688 # 1
BadPix(0.03) 3.6084 # 1
BadPix(0.01) 17.7493 # 1

Methods