Search Results for author:
Found 19 papers, 12 papers with code
DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation
Story visualization, the task of creating visual narratives from textual descriptions, has seen progress with text-to-image generation models.
HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation
Notably, we introduce a rule-based camera trajectory generation method, enabling the synthetic pipeline to incorporate diverse and precise camera motion annotation, which can rarely be found in real-world data.
Live2Diff: Live Stream Translation via Uni-directional Attention in Video Diffusion Models
State-of-the-art video diffusion models leverage bi-directional temporal attention to model the correlations between the current frame and all the surrounding (i. e. including future) frames, which hinders them from processing streaming videos.
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds
Meanwhile, the temporal controller incorporates an onset detector and a timestampbased adapter to achieve precise audio-video alignment.
StyleShot: A Snapshot on Any Style
In this paper, we show that, a good style representation is crucial and sufficient for generalized style transfer without test-time tuning.
Ranked #1 on
Style Transfer
on StyleBench
Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Language
Diffusion models can generate realistic and diverse images, potentially facilitating data availability for data-intensive perception tasks.
MotionBooth: Motion-Aware Customized Text-to-Video Generation
In this work, we present MotionBooth, an innovative framework designed for animating customized subjects with precise control over both object and camera movements.
Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion Prior
Capturing High Dynamic Range (HDR) scenery using 8-bit cameras often suffers from over-/underexposure, loss of fine details due to low bit-depth compression, skewed color distributions, and strong noise in dark areas.
Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text
Creating and animating 3D biped cartoon characters is crucial and valuable in various applications.
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models
Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles.
A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting
Second, we demonstrate the versatility of the task prompt in PowerPaint by showcasing its effectiveness as a negative prompt for object removal.
Degradation-Guided Meta-Restoration Network for Blind Super-Resolution
The extractor estimates the degradations in LR inputs and guides the meta-restoration modules to predict restoration parameters for different degradations on-the-fly.
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
To enable VL pre-training, we jointly optimize the HD-VILA model by a hybrid Transformer that learns rich spatiotemporal features, and a multimodal Transformer that enforces interactions of the learned video features with diversified texts.
Ranked #17 on
Video Retrieval
on MSR-VTT
Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers
Given a sequence of style tokens, the TokenGAN is able to control the image synthesis by assigning the styles to the content tokens by attention mechanism with a Transformer.
3D Human Body Reshaping with Anthropometric Modeling
First, we calculate full-body anthropometric parameters from limited user inputs by imputation technique, and thus essential anthropometric parameters for 3D body reshaping can be obtained.
Aggregated Contextual Transformations for High-Resolution Image Inpainting
For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task.
Ranked #11 on
Image Inpainting
on Places2
Learning Semantic-aware Normalization for Generative Adversarial Networks
Such a model disentangles latent factors according to the semantic of feature channels by channel-/group- wise fusion of latent codes and feature channels.
Learning Joint Spatial-Temporal Transformations for Video Inpainting
In this paper, we propose to learn a joint Spatial-Temporal Transformer Network (STTN) for video inpainting.
Ranked #5 on
Seeing Beyond the Visible
on KITTI360-EX
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
As the missing content can be filled by attention transfer from deep to shallow in a pyramid fashion, both visual and semantic coherence for image inpainting can be ensured.
