Video Prediction
163 papers with code • 19 benchmarks • 24 datasets
Video Prediction is the task of predicting future frames given past video frames.
Gif credit: MAGVIT
Source: Photo-Realistic Video Prediction on Natural Videos of Largely Changing Frames
Benchmarks
These leaderboards are used to track progress in Video Prediction
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Video Prediction models and implementationsDatasets
MNIST
Cityscapes
KITTI
Kinetics
Human3.6M
DAVIS
KTH
DAVIS 2017
Something-Something V2
Moving MNIST
Most implemented papers
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
coxlab/prednet
•
•
Here, we explore prediction of future frames in a video sequence as an unsupervised learning rule for learning about the structure of the visual world.
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
ndrplz/ConvLSTM_pytorch
•
•
NeurIPS 2015
The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time.
Video-to-Video Synthesis
NVIDIA/vid2vid
•
•
NeurIPS 2018
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Yunbo426/predrnn-pp
•
•
ICML 2018
We present PredRNN++, an improved recurrent network for video predictive learning.
Deep multi-scale video prediction beyond mean square error
dyelax/Adversarial_Video_Generation
•
•
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.
The "something something" video database for learning and evaluating visual common sense
jayleicn/singularity
•
•
ICCV 2017
Neural networks trained on datasets such as ImageNet have led to major advances in visual object classification.
Learning a Driving Simulator
commaai/research
•
•
Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
Hzzone/Precipitation-Nowcasting
•
•
NeurIPS 2017
To address these problems, we propose both a new model and a benchmark for precipitation nowcasting.
Stochastic Adversarial Video Prediction
alexlee-gk/video_prediction
•
•
ICLR 2019
However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.
Video Diffusion Models
lucidrains/video-diffusion-pytorch
•
•
Generating temporally coherent high fidelity video is an important milestone in generative modeling research.
