315 papers with code • 8 benchmarks • 26 datasets
Optical Flow Estimation is the problem of finding pixel-wise motions between consecutive images.
Learning to represent videos is a very challenging task both algorithmically and computationally.
We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences and extra images to surpass state-of-the-art performance on core computer vision tasks.
We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective.
Ranked #2 on Optical Flow Estimation on Sintel Clean unsupervised
The proposed model then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels for synthesizing the output frame.
Ranked #3 on Video Frame Interpolation on X4K1000FPS
Finally, the two input images are warped and linearly fused to form each intermediate frame.
Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods.
RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes.
Ranked #2 on Optical Flow Estimation on Sintel-final