Search Results for author:
Found 56 papers, 37 papers with code
Look at Boundary: A Boundary-Aware Face Alignment Algorithm
By utilising boundary information of 300-W dataset, our method achieves 3. 92% mean error with 0. 39% failure rate on COFW dataset, and 1. 25% mean error on AFLW-Full dataset.
Ranked #4 on
Face Alignment
on AFLW-19
(using extra training data)
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.
MotionBERT: A Unified Perspective on Learning Human Motion Representations
We present a unified perspective on tackling various human-centric video tasks by learning human motion representations from large-scale and heterogeneous data resources.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(using extra training data)
Text2Human: Text-Driven Controllable Human Image Generation
In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation.
Focal Frequency Loss for Image Reconstruction and Synthesis
In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further.
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
The quality of generated videos outperforms those in existing datasets, validated by user studies.
DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases.
CelebV-Text: A Large-Scale Facial Text-Video Dataset
This paper presents CelebV-Text, a large-scale, diverse, and high-quality dataset of facial text-video pairs, to facilitate research on facial text-to-video generation tasks.
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
Text2Performer: Text-Driven Human Video Generation
In this work, we present Text2Performer to generate vivid human videos with articulated motions from texts.
Audio-Driven Emotional Video Portraits
In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation.
Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.
3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping
We present 3DHumanGAN, a 3D-aware generative adversarial network that synthesizes photorealistic images of full-body humans with consistent appearances under different view-angles and body-poses.
OrthoPlanes: A Novel Representation for Better 3D-Awareness of GANs
We present a new method for generating realistic and view-consistent images with fine geometry from 2D image collections.
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
It is a large-scale digital library for head avatars with three key attributes: 1) High Fidelity: all subjects are captured by 60 synchronized, high-resolution 2K cameras in 360 degrees.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
ReenactGAN: Learning to Reenact Faces via Boundary Transfer
A transformer is subsequently used to adapt the boundary of source face to the boundary of target face.
TAM: Temporal Adaptive Module for Video Recognition
Video data is with complex temporal dynamics due to various factors such as camera motion, speed variation, and different activities.
Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation
Facial landmark detection, or face alignment, is a fundamental task that has been extensively studied.
Ranked #18 on
Face Alignment
on WFLW
Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis
Specifically, we compress the light fields for novel view human rendering as conditional implicit neural radiance fields from both geometry and appearance aspects.
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
In this study, we highlight the importance of interaction in a dual-space GAN for more controllable editing.
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets.
Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis
In this paper, we present a spatial-temporal compression framework, \textbf{Fast-Vid2Vid}, which focuses on data aspects of generative models.
StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3
Notably, StyleFaceV is capable of generating realistic $1024\times1024$ face videos even without high-resolution training videos.
MonoHuman: Animatable Human Neural Field from Monocular Video
Recent works propose to graft a deformation network into the NeRF to further model the dynamics of the human neural field for animating vivid human motions.
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation
To enhance the quality of synthesized gestures, we develop a contrastive learning strategy based on audio-text alignment for better audio representations.
Ranked #3 on
Gesture Generation
on TED Gesture Dataset
FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos
A structure predictor is proposed to predict the missing face structural information temporally, which serves as a geometry prior.
ReliTalk: Relightable Talking Portrait Generation from a Single Video
Our key insight is to decompose the portrait's reflectance from implicitly learned audio-driven facial normals and images.
Everything's Talkin': Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation
A holistic human dataset inevitably has insufficient and low-resolution information on local parts.
Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection
Generic event boundary detection is an important yet challenging task in video understanding, which aims at detecting the moments where humans naturally perceive event boundaries.
Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing
This paper focuses on the weakly-supervised audio-visual video parsing task, which aims to recognize all events belonging to each modality and localize their temporal boundaries.
VLG: General Video Recognition with Web Textual Knowledge
Our VLG is first pre-trained on video and language datasets to learn a shared feature space, and then devises a flexible bi-modal attention head to collaborate high-level semantic concepts under different settings.
Disentangling Content and Style via Unsupervised Geometry Distillation
It is challenging to disentangle an object into two orthogonal spaces of content and style since each can influence the visual observation differently and unpredictably.
TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks.
Make a Face: Towards Arbitrary High Fidelity Face Manipulation
Recent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to low-resolution and lack of diversity.
Everybody's Talkin': Let Me Talk as You Want
The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.
AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection
However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation.
Pareidolia Face Reenactment
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
Unsupervised Disentangling Structure and Appearance
It is challenging to disentangle an object into two orthogonal spaces of structure and appearance since each can influence the visual observation in a different and unpredictable way.
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i. e., motion, structure, and view angle.
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
Moreover, to enable portrait rendering in one unified neural radiance field, a Torso Deformation module is designed to stabilize the large-scale non-rigid torso motions.
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
Although significant progress has been made to audio-driven talking face generation, existing methods either neglect facial emotion or cannot be applied to arbitrary subjects.
Submission to Generic Event Boundary Detection Challenge@CVPR 2022: Local Context Modeling and Global Boundary Decoding Approach
Local context modeling sub-network is proposed to perceive diverse patterns of generic event boundaries, and it generates powerful video representations and reliable boundary confidence.
Audio-Driven Co-Speech Gesture Video Generation
Our key insight is that the co-speech gestures can be decomposed into common motion patterns and subtle rhythmic dynamics.
HyperStyle3D: Text-Guided 3D Portrait Stylization via Hypernetworks
While the use of 3D-aware GANs bypasses the requirement of 3D data, we further alleviate the necessity of style images with the CLIP model being the stylization guidance.
Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis
State-of-the-art methods in this direction typically consider building separate networks for these two tasks (i. e., view synthesis and editing).
Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis
Existing automated dubbing methods are usually designed for Professionally Generated Content (PGC) production, which requires massive training data and training time to learn a person-specific audio-video mapping.
Innovative Digital Storytelling with AIGC: Exploration and Discussion of Recent Advances
Digital storytelling, as an art form, has struggled with cost-quality balance.
Parameterization-driven Neural Implicit Surfaces Editing
In this paper, we present a novel neural algorithm to parameterize neural implicit surfaces to simple parametric domains, such as spheres, cubes, or polycubes, thereby facilitating visualization and various editing tasks.
PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation
We first propose a novel score function, Denoised Score Distillation (DSD), which directly modifies the SDS by introducing negative gradient components to iteratively correct the gradient direction and generate high-quality textures.
CosmicMan: A Text-to-Image Foundation Model for Humans
We present CosmicMan, a text-to-image foundation model specialized for generating high-fidelity human images.
