Image to Video Generation
14 papers with code • 0 benchmarks • 0 datasets
Image to Video Generation refers to the task of generating a sequence of video frames based on a single still image or a set of still images. The goal is to produce a video that is coherent and consistent in terms of appearance, motion, and style, while also being temporally consistent, meaning that the generated video should look like a coherent sequence of frames that are temporally ordered. This task is typically tackled using deep generative models, such as Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs), that are trained on large datasets of videos. The models learn to generate plausible video frames that are conditioned on the input image, as well as on any other auxiliary information, such as a sound or text track.
Benchmarks
These leaderboards are used to track progress in Image to Video Generation
Most implemented papers
Collaborative Neural Rendering using Anime Character Sheets
Drawing images of characters with desired poses is an essential but laborious task in anime production.
Video Generation from Single Semantic Label Map
This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides a good balance between flexibility and quality in the generation process.
Lifespan Age Transformation Synthesis
royorel/Lifespan_Age_Transformation_Synthesis
•
•
ECCV 2020
Most existing aging methods are limited to changing the texture, overlooking transformations in head shape that occur during the human aging and growth process.
SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields
astra-vision/SceneRF
•
•
ICCV 2023
3D reconstruction from a single 2D image was extensively covered in the literature but relies on depth supervision at training time, which limits its applicability.
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
stability-ai/generative-models
•
•
None 2023
We then explore the impact of finetuning our base model on high-quality data and train a text-to-video model that is competitive with closed-source video generation.
Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts
Despite recent advances in image-to-video generation, better controllability and local animation are less explored.
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
We consider the problem of image-to-video translation, where an input image is translated into an output video containing motions of a single object.
Make It Move: Controllable Image-to-Video Generation with Text Descriptions
With both controllable appearance and motion, TI2V aims at generating videos from a static image and a text description.
A Method for Animating Children's Drawings of the Human Figure
Children's drawings have a wonderful inventiveness, creativity, and variety to them.
Conditional Image-to-Video Generation with Latent Flow Diffusion Models
nihaomiao/cvpr23_lfdm
•
•
CVPR 2023
In this paper, we propose an approach for cI2V using novel latent flow diffusion models (LFDM) that synthesize an optical flow sequence in the latent space based on the given condition to warp the given image.
