Video Prediction
State-of-the-art leaderboards
Greatest papers with code
Unsupervised Learning for Physical Interaction through Video Prediction
NeurIPS 2016
• tensorflow/models
• 
A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment.
Video-to-Video Synthesis
NeurIPS 2018
• NVIDIA/vid2vid
• 
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e. g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video.
SEMANTIC SEGMENTATION VIDEO PREDICTION VIDEO-TO-VIDEO SYNTHESIS
Learning a Driving Simulator
• commaai/research
•
Comma. ai's approach to Artificial Intelligence for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road.
Deep multi-scale video prediction beyond mean square error
• dyelax/Adversarial_Video_Generation
• 
Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics.
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• coxlab/prednet
•
Here, we explore prediction of future frames in a video sequence as an unsupervised learning rule for learning about the structure of the visual world.
Stochastic Adversarial Video Prediction
ICLR 2019
• alexlee-gk/video_prediction
•
However, learning to predict raw future observations, such as frames in a video, is exceedingly challenging -- the ambiguous nature of the problem can cause a naively designed model to average together possible futures into a single, blurry prediction.
Predicting Deeper into the Future of Semantic Segmentation
ICCV 2017
• facebookresearch/SegmPred
•
The ability to predict and therefore to anticipate the future is an important attribute of intelligence.
AUTONOMOUS DRIVING DECISION MAKING OPTICAL FLOW ESTIMATION SCENE UNDERSTANDING SEMANTIC SEGMENTATION VIDEO PREDICTION
Prediction Under Uncertainty with Error-Encoding Networks
In this work we introduce a new framework for performing temporal predictions in the presence of uncertainty.
Predicting Future Instance Segmentation by Forecasting Convolutional Features
ECCV 2018 • facebookresearch/instpred
We apply the "detection head'" of Mask R-CNN on the predicted features to produce the instance segmentation of future frames.
INSTANCE SEGMENTATION OPTICAL FLOW ESTIMATION SEMANTIC SEGMENTATION VIDEO PREDICTION
The "something something" video database for learning and evaluating visual common sense
ICCV 2017
• TwentyBN/smth-smth-v2-baseline-with-models
•
Neural networks trained on datasets such as ImageNet have led to major advances in visual object classification.
COMMON SENSE REASONING OBJECT CLASSIFICATION VIDEO PREDICTION