Search Results for author:
Found 26 papers, 19 papers with code
Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation
Diffusion models have proven to be highly effective in image and video generation; however, they still face composition challenges when generating images of varying sizes due to single-scale training data.
VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
Based on this stronger coupling, we shift the distribution to higher quality without motion degradation by finetuning spatial modules with high-quality images, resulting in a generic high-quality video model.
Ranked #1 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Therefore, this paper presents MotionCtrl, a unified and flexible motion controller for video generation designed to effectively and independently control camera and object motion.
StyleCrafter: Enhancing Stylized Text-to-Video Generation with Style Adapter
To address these challenges, we introduce StyleCrafter, a generic method that enhances pre-trained T2V models with a style control adapter, enabling video generation in any style by providing a reference image.
Sketch Video Synthesis
Understanding semantic intricacies and high-level concepts is essential in image sketch generation, and this challenge becomes even more formidable when applied to the domain of videos.
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
The I2V model is designed to produce videos that strictly adhere to the content of the provided reference image, preserving its content, structure, and style.
Ranked #3 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
With the availability of large-scale video datasets and the advances of diffusion models, text-driven video generation has achieved substantial progress.
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Animating a still image offers an engaging visual experience.
ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models
Our work also suggests that a pre-trained diffusion model trained on low-resolution images can be directly used for high-resolution visual generation without further tuning, which may provide insights for future research on ultra-high-resolution image and video synthesis.
Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure.
Taming Reversible Halftoning via Predictive Luminance
Furthermore, to tackle the conflicts between the blue-noise quality and restoration accuracy in our novel base method, we proposed a predictor-embedded approach to offload predictable information from the network, which in our case is the luminance information resembling from the halftone pattern.
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance
Our method, dubbed Make-Your-Video, involves joint-conditional video generation using a Latent Diffusion Model that is pre-trained for still image synthesis and then promoted for video generation with the introduction of temporal modules.
TaleCrafter: Interactive Story Visualization with Multiple Characters
Accurate Story visualization requires several necessary elements, such as identity consistency across frames, the alignment between plain text and visual content, and a reasonable layout of objects in images.
CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying
Thanks to the proposed structure, we only encode the high-resolution image in a relatively low resolution for larger reception field capturing.
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
In this paper, we propose to cast speech-driven facial animation as a code query task in a finite proxy space of the learned codebook, which effectively promotes the vividness of the generated motions by reducing the cross-modal mapping uncertainty.
Ranked #4 on
3D Face Animation
on BEAT2
Disentangled Image Colorization via Global Anchors
Our key insight is that several carefully located anchors could approximately represent the color distribution of an image, and conditioned on the anchor colors, we can predict the image color in a deterministic manner by utilizing internal correlation.
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild
Our system disentangles this objective into three sequential tasks: (1) face video generation with a canonical expression; (2) audio-driven lip-sync; and (3) face enhancement for improving photo-realism.
PalGAN: Image Colorization with Palette Generative Adversarial Networks
Multimodal ambiguity and color bleeding remain challenging in colorization.
Screentone-Preserved Manga Retargeting
Fortunately, the rescaled manga shares the same region-wise screentone correspondences with the original manga, which enables us to simplify the screentone synthesis problem as an anchor-based proposals selection and rearrangement problem.
Invertible Tone Mapping with Selectable Styles
In this paper, we propose an invertible tone mapping method that converts the multi-exposure HDR to a true LDR (8-bit per color channel) and reserves the capability to accurately restore the original HDR from this {\em invertible LDR}.
Exploiting Aliasing for Manga Restoration
First, we predict the target resolution from the degraded manga via the Scale Estimation Network (SE-Net) with spatial voting scheme.
A Learned Compact and Editable Light Field Representation
Light fields are 4D scene representation typically structured as arrays of views, or several directional samples per pixel in a single view.
Deep Halftoning With Reversible Binary Pattern
Existing halftoning algorithms usually drop colors and fine details when dithering color images with binary dot patterns, which makes it extremely difficult to recover the original information.
Enhance Convolutional Neural Networks with Noise Incentive Block
Noise Incentive Block (NIB), which serves as a generic plug-in for any CNN generation model.
Mononizing Binocular Videos
This paper presents the idea ofmono-nizingbinocular videos and a frame-work to effectively realize it.
Image and Video Processing Graphics
Line-Based Multi-Label Energy Optimization for Fisheye Image Rectification and Calibration
Fisheye image rectification and estimation of intrinsic parameters for real scenes have been addressed in the literature by using line information on the distorted images.
