Computational Tradeoffs in Image Synthesis: Diffusion, Masked-Token, and Next-Token Prediction

no code implementations • 21 May 2024 • Maciej Kilian, Varun Jampani, Luke Zettlemoyer

We find that token prediction methods, led by next-token prediction, significantly outperform diffusion on prompt following.

Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets

2 code implementations • None 2023 • Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, Robin Rombach

We then explore the impact of finetuning our base model on high-quality data and train a text-to-video model that is competitive with closed-source video generation.

Image Generation Image to Video Generation