Optical Flow Estimation

444 papers with code • 9 benchmarks • 29 datasets

Optical Flow Estimation is the problem of finding pixel-wise motions between consecutive images.

Approaches for optical flow estimation include correlation-based, block-matching, feature tracking, energy-based, and more recently gradient-based.

Further readings:

Definition source: Devon: Deformable Volume Network for Learning Optical Flow

Image credit: Optical Flow Estimation


Use these libraries to find Optical Flow Estimation models and implementations

Most implemented papers

FlowNet: Learning Optical Flow with Convolutional Networks

msracver/Deep-Feature-Flow ICCV 2015

Optical flow estimation has not been among the tasks where CNNs were successful.

PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume

NVlabs/PWC-Net CVPR 2018

It then uses the warped features and features of the first image to construct a cost volume, which is processed by a CNN to estimate the optical flow.

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

hzwer/Arxiv2020-RIFE 12 Nov 2020

We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for Video Frame Interpolation (VFI).

Optical Flow Estimation using a Spatial Pyramid Network

anuragranj/spynet CVPR 2017

We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning.

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

NVIDIA/flownet2-pytorch CVPR 2017

Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods.

Semantic Flow for Fast and Accurate Scene Parsing

donnyyou/torchcv ECCV 2020

A common practice to improve the performance is to attain high resolution feature maps with strong semantic representation.

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

princeton-vl/RAFT ECCV 2020

RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes.

Perceiver IO: A General Architecture for Structured Inputs & Outputs

deepmind/deepmind-research ICLR 2022

A central goal of machine learning is the development of systems that can solve many problems in as many data domains as possible.

Two-Stream Convolutional Networks for Action Recognition in Videos

woodfrog/ActionRecognition NeurIPS 2014

Our architecture is trained and evaluated on the standard video actions benchmarks of UCF-101 and HMDB-51, where it is competitive with the state of the art.

Video Frame Interpolation via Adaptive Separable Convolution

sniklaus/sepconv-slomo ICCV 2017

Our method develops a deep fully convolutional neural network that takes two input frames and estimates pairs of 1D kernels for all pixels simultaneously.