Search Results for author:
Found 30 papers, 11 papers with code
X-Dancer: Expressive Music to Human Dance Video Generation
As its core, we introduce a unified transformer-diffusion framework, featuring an autoregressive transformer model that synthesize extended and music-synchronized token sequences for 2D body, head and hands poses, which then guide a diffusion model to produce coherent and realistic dance video frames.
X-Dyna: Expressive Dynamic Human Image Animation
At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details.
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection
While LoRA, a popular parameter-efficient method, reduces memory usage, it often suffers from suboptimal performance due to the constraints of low-rank updates.
ID-Patch: Robust ID Association for Group Photo Personalization
Existing methods suffer from limitations such as the reliance on segmentation models, increased runtime, or a high probability of ID leakage.
High Quality Human Image Animation using Regional Supervision and Motion Blur Condition
Although generating reasonable results, existing methods often overlook the need for regional supervision in crucial areas such as the face and hands, and neglect the explicit modeling for motion blur, leading to unrealistic low-quality synthesis.
Learning Feature-Preserving Portrait Editing from Generated Pairs
Portrait editing is challenging for existing techniques due to difficulties in preserving subject features like identity.
X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
We propose X-Portrait, an innovative conditional diffusion model tailored for generating expressive and temporally coherent portrait animation.
DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generation
The generation of emotional talking faces from a single portrait image remains a significant challenge.
DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis
The rendering view is then manipulated with a novel conditional control module that interprets the camera pose by watching a condition image of a crossed subject from the same view.
FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features
The proposed FG-Net achieves a strong generalization ability for heatmap-based AU detection thanks to the generalizable and semantic-rich features extracted from the pre-trained generative model.
OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis
We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D heads with compelling dynamic details under full disentangled control over camera poses, facial expressions, head shapes, articulated neck and jaw poses.
AgileGAN3D: Few-Shot 3D Portrait Stylization by Augmented Transfer Learning
Capitalizing on the recent advancement of 3D-aware GAN models, we perform \emph{guided transfer learning} on a pretrained 3D GAN generator to produce multi-view-consistent stylized renderings.
PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360$^{\circ}$
We propose PanoHead, the first 3D-aware generative model that enables high-quality view-consistent image synthesis of full heads in $360^\circ$ with diverse appearance and detailed geometry using only in-the-wild unstructured images for training.
PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360deg
We propose PanoHead, the first 3D-aware generative model that enables high-quality view-consistent image synthesis of full heads in 360deg with diverse appearance and detailed geometry using only in-the-wild unstructured images for training.
AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging
We propose a novel self-supervised learning framework to create high-quality stylized 3D avatars with a mix of continuous and discrete parameters.
Using Augmented Face Images to Improve Facial Recognition Tasks
We present a framework that uses GAN-augmented images to complement certain specific attributes, usually underrepresented, for machine learning model training.
High-Quality Pluralistic Image Completion via Code Shared VQGAN
In this work, we present a novel framework for pluralistic image completion that can achieve both high quality and diversity at much faster inference speed.
AgileGAN: stylizing portraits by inversion-consistent transfer learning
While substantial progress has been made in automated stylization, generating high quality stylistic portraits is still a challenge, and even the recent popular Toonify suffers from several artifacts when used on real input images.
DynOcc: Learning Single-View Depth from Dynamic Occlusion Cues
Recently, significant progress has been made in single-view depth estimation thanks to increasingly large and diverse depth datasets.
HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures
We present HoliCity, a city-scale 3D dataset with rich structural information.
Dynamic Kernel Distillation for Efficient Pose Estimation in Videos
Existing video-based human pose estimation methods extensively apply large networks onto every frame in the video to localize body joints, which suffer high computational cost and hardly meet the low-latency requirement in realistic applications.
Ranked #4 on
2D Human Pose Estimation
on JHMDB (2D poses only)
Task-Assisted Domain Adaptation with Anchor Tasks
Our key idea to improve domain adaptation is to introduce a separate anchor task (such as facial landmarks) whose annotations can be obtained at no cost or are already available on both synthetic and real datasets.
Transformable Bottleneck Networks
We propose a novel approach to performing fine-grained 3D manipulation of image content via a convolutional neural network, which we call the Transformable Bottleneck Network (TBN).
Extreme Relative Pose Estimation for RGB-D Scans via Scene Completion
In particular, instead of only performing scene completion from each individual scan, our approach alternates between relative pose estimation and scene completion.
Deep Volumetric Video From Very Sparse Multi-View Performance Capture
We present a deep learning-based volumetric capture approach for performance capture using a passive and highly sparse multi-view capture system.
Deep Generative Modeling for Scene Synthesis via Hybrid Representations
We show a principled way to train this model by combining discriminator losses for both a 3D object arrangement representation and a 2D image-based representation.
StarMap for Category-Agnostic Keypoint and Viewpoint Estimation
Existing methods define semantic keypoints separately for each category with a fixed number of semantic labels in fixed indices.
Ranked #2 on
Keypoint Detection
on Pascal3D+
Unsupervised Domain Adaptation for 3D Keypoint Estimation via View Consistency
In this paper, we introduce a novel unsupervised domain adaptation technique for the task of 3D keypoint prediction from a single depth scan or image.
AutoScaler: Scale-Attention Networks for Visual Correspondence
We propose AutoScaler, a scale-attention network to explicitly optimize this trade-off in visual correspondence tasks.
Wide-Baseline Hair Capture Using Strand-Based Refinement
We propose a novel algorithm to reconstruct the 3D geometry of human hairs in wide-baseline setups using strand-based refinement.
