Implicit Stacked Autoregressive Model for Video Prediction

14 Mar 2023  ·  Minseok Seo, Hakjin Lee, Doyi Kim, Junghoon Seo ·

Future frame prediction has been approached through two primary methods: autoregressive and non-autoregressive. Autoregressive methods rely on the Markov assumption and can achieve high accuracy in the early stages of prediction when errors are not yet accumulated. However, their performance tends to decline as the number of time steps increases. In contrast, non-autoregressive methods can achieve relatively high performance but lack correlation between predictions for each time step. In this paper, we propose an Implicit Stacked Autoregressive Model for Video Prediction (IAM4VP), which is an implicit video prediction model that applies a stacked autoregressive method. Like non-autoregressive methods, stacked autoregressive methods use the same observed frame to estimate all future frames. However, they use their own predictions as input, similar to autoregressive methods. As the number of time steps increases, predictions are sequentially stacked in the queue. To evaluate the effectiveness of IAM4VP, we conducted experiments on three common future frame prediction benchmark datasets and weather\&climate prediction benchmark datasets. The results demonstrate that our proposed model achieves state-of-the-art performance.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Video Prediction Human3.6M IAM4VP SSIM 0.942 # 1
Video Prediction Moving MNIST IAM4VPx5 MSE 15.3 # 2
MAE 49.2 # 1
SSIM 0.966 # 4
Weather Forecasting SEVIR IAM4VP MSE 2.9371 # 1
mCSI 0.4607 # 1


No methods listed for this paper. Add relevant methods here