Video Prediction

183 papers with code • 19 benchmarks • 24 datasets

Video Prediction is the task of predicting future frames given past video frames.

Gif credit: MAGVIT

Source: Photo-Realistic Video Prediction on Natural Videos of Largely Changing Frames


Use these libraries to find Video Prediction models and implementations

Most implemented papers

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

ndrplz/ConvLSTM_pytorch NeurIPS 2015

The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time.

Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning

coxlab/prednet 25 May 2016

Here, we explore prediction of future frames in a video sequence as an unsupervised learning rule for learning about the structure of the visual world.

Video-to-Video Synthesis

NVIDIA/vid2vid NeurIPS 2018

We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.

MogaNet: Multi-order Gated Aggregation Network

Westlake-AI/openmixup 7 Nov 2022

Notably, MogaNet hits 80. 0\% and 87. 8\% accuracy with 5. 2M and 181M parameters on ImageNet-1K, outperforming ParC-Net and ConvNeXt-L, while saving 59\% FLOPs and 17M parameters, respectively.

Deep multi-scale video prediction beyond mean square error

dyelax/Adversarial_Video_Generation 17 Nov 2015

Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.

The "something something" video database for learning and evaluating visual common sense

jayleicn/singularity ICCV 2017

Neural networks trained on datasets such as ImageNet have led to major advances in visual object classification.

Learning a Driving Simulator

commaai/research 3 Aug 2016

Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.

Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model

Hzzone/Precipitation-Nowcasting NeurIPS 2017

To address these problems, we propose both a new model and a benchmark for precipitation nowcasting.

Stochastic Adversarial Video Prediction

alexlee-gk/video_prediction ICLR 2019

However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.