Video Prediction

90 papers with code • 9 benchmarks • 14 datasets

Video Prediction is the task of predicting future frames given past video frames.

Source: Photo-Realistic Video Prediction on Natural Videos of Largely Changing Frames

Greatest papers with code

Unsupervised Learning for Physical Interaction through Video Prediction

tensorflow/models NeurIPS 2016

A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment.

Video Prediction

Unsupervised Learning of Object Structure and Dynamics from Videos

google-research/google-research NeurIPS 2019

Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning.

Action Recognition Continuous Control +2

VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation

tensorflow/tensor2tensor ICLR 2020

Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions.

Predict Future Video Frames Video Generation

Model-Based Reinforcement Learning for Atari

tensorflow/tensor2tensor 1 Mar 2019

We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.

Atari Games Model-based Reinforcement Learning +1

Video-to-Video Synthesis

NVIDIA/vid2vid NeurIPS 2018

We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.

Semantic Segmentation Video Prediction +1

Learning a Driving Simulator

commaai/research 3 Aug 2016

Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.

Self-Driving Cars Video Prediction

SDCNet: Video Prediction Using Spatially-Displaced Convolution

NVIDIA/semantic-segmentation 2 Nov 2018

We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.

Optical Flow Estimation SSIM +1

SDC-Net: Video prediction using spatially-displaced convolution

NVIDIA/semantic-segmentation ECCV 2018

We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.

Optical Flow Estimation SSIM +1

Scalable Gradients for Stochastic Differential Equations

google-research/torchsde 5 Jan 2020

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.

Motion Capture Variational Inference +1

Deep multi-scale video prediction beyond mean square error

dyelax/Adversarial_Video_Generation 17 Nov 2015

Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.

Optical Flow Estimation Video Prediction