Towards Non-Parametric Models for Confidence Aware Video Prediction on Smooth Dynamics

The ability to envision future states is crucial to informed decision making while interacting with dynamic environments. With vision providing an information rich sensing modality, the field of video prediction has garnered a lot of attention in pursuit of this ability. Current state of the art methods rely on neural network based models for prediction. Though often accurate, these methods require large amounts of training data that are often unavailable when encountering unknown environments. Furthermore, the predictive accuracy of such methods breaks down without warning, when tested on data far outside their training distribution. This problem is exacerbated by the fact that these networks can be prohibitively expensive to update with recent data acquired online. To overcome these drawbacks we use non-parametric models to take a probabilistic approach to video prediction for problems with little training data. We generate probability distributions over sequentially predicted images and propagate our uncertainty through time to generate a confidence metric for our predictions. We use non-parametric Gaussian Process models for their data efficiency and ability to readily incorporate new training data online. To showcase our method we successfully predict future frames of a smooth fluid simulation environment. In this paper we propose a non-parametric method using Gaussian Process models to propagate probability distributions over sequentially predicted images for confidence aware video prediction with little training.