Disentangling Multiple Features in Video Sequences using Gaussian Processes in Variational Autoencoders

ECCV 2020 Sarthak BhagatShagun UppalZhuyun YinNengli Lim

We introduce MGP-VAE (Multi-disentangled-features Gaussian Processes Variational AutoEncoder), a variational autoencoder which uses Gaussian processes (GP) to model the latent space for the unsupervised learning of disentangled representations in video sequences. We improve upon previous work by establishing a framework by which multiple features, static or dynamic, can be disentangled... (read more)

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Video Prediction Colored dSprites MGP-VAE (with geodesic loss) MSE 4.5 # 1
Video Prediction Moving MNIST MGP-VAE MSE 25.4 # 2
Video Prediction Moving MNIST MGP-VAE (with geodesic loss) MSE 18.5 # 1
Video Prediction Sprites MGP-VAE (with geodesic loss) MSE 61.6 # 1

Methods used in the Paper