Video Prediction
90 papers with code • 9 benchmarks • 14 datasets
Video Prediction is the task of predicting future frames given past video frames.
Source: Photo-Realistic Video Prediction on Natural Videos of Largely Changing Frames
Benchmarks
| Trend | Dataset | Best Model | Paper Title | Paper | Code | Compare |
|---|---|---|---|---|---|---|
|
||||||
|
||||||
|
||||||
|
||||||
|
||||||
|
||||||
|
||||||
|
||||||
|
Datasets
MNIST
Kinetics
Human3.6M
KTH
Moving MNIST
YouTube-8M
Kinetics-600
Sprites
PHYRE
Robotic Pushing
Greatest papers with code
Unsupervised Learning for Physical Interaction through Video Prediction
tensorflow/models
•
•
NeurIPS 2016
A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment.
Unsupervised Learning of Object Structure and Dynamics from Videos
google-research/google-research
•
•
NeurIPS 2019
Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning.
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
tensorflow/tensor2tensor
•
•
ICLR 2020
Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions.
Model-Based Reinforcement Learning for Atari
We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting.
Ranked #12 on
Atari Games
on Atari games
Video-to-Video Synthesis
NVIDIA/vid2vid
•
•
NeurIPS 2018
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
Learning a Driving Simulator
commaai/research
•
•
Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.
SDCNet: Video Prediction Using Spatially-Displaced Convolution
NVIDIA/semantic-segmentation
•
•
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
SDC-Net: Video prediction using spatially-displaced convolution
NVIDIA/semantic-segmentation
•
•
ECCV 2018
We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.
Ranked #1 on
Video Prediction
on YouTube-8M
Scalable Gradients for Stochastic Differential Equations
The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.
Ranked #1 on
Video Prediction
on CMU Mocap-2
Deep multi-scale video prediction beyond mean square error
dyelax/Adversarial_Video_Generation
•
•
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.