Search Results for author:
Found 52 papers, 21 papers with code
ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer
To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization, thereby improving the naturalness and fluency of zero-shot motion generation for unseen speech inputs.
ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping
In this paper, we introduce ManiVideo, a novel method for generating consistent and temporally coherent bimanual hand-object manipulation videos from given motion sequences of hands and objects.
Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts
This paper introduces Stereo-Talker, a novel one-shot audio-driven human video synthesis system that generates 3D talking videos with precise lip synchronization, expressive body gestures, temporally consistent photo-realistic quality, and continuous viewpoint control.
Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns
We propose Lodge++, a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre.
Ranked #2 on
Motion Synthesis
on FineDance
ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusion
At each step of the denoising process, we incorporate the current hand pose residual as a refinement target into the network, guiding the network to correct inaccurate hand poses.
HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Models
In this paper, we extend the ability of controllable generative models for a more comprehensive hand mesh recovery task: direct hand mesh generation, inpainting, reconstruction, and fitting in a single framework, which we name as Holistic Hand Mesh Recovery (HHMR).
Ranked #7 on
3D Hand Pose Estimation
on FreiHAND
OmniHands: Towards Robust 4D Hand Mesh Recovery via A Versatile Transformer
The results on in-the-wild videos and real-world scenarios demonstrate the superior performances of our approach for interactive hand reconstruction.
LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer
In the single-layer reconstruction stage, we propose a series of geometric constraints to reconstruct smooth surfaces and simultaneously obtain the segmentation between body and clothing.
Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives
In contrast, the second-stage is the local diffusion, which parallelly generates detailed motion sequences under the guidance of the dance primitives and choreographic rules.
Ranked #3 on
Motion Synthesis
on FineDance
STAF: 3D Human Mesh Recovery from Video with Spatio-Temporal Alignment Fusion
This method can remarkably improve the smoothness of recovery results from video.
Ranked #53 on
3D Human Pose Estimation
on MPI-INF-3DHP
(using extra training data)
Ins-HOI: Instance Aware Human-Object Interactions Recovery
To address this, we further propose a complementary training strategy that leverages synthetic data to introduce instance-level shape priors, enabling the disentanglement of occupancy fields for different instances.
FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions
By adopting a divide-and-conquer strategy, we propose a new framework named Fine-Grained Human Motion Diffusion Model (FG-MDM) for zero-shot human motion generation.
HHAvatar: Gaussian Head Avatar with Dynamic Hairs
To address the problem of dynamic hair modeling, we introduce a hybrid head model into our avatar representation based Gaussian Head Avatar and a training method that considers timing information and an occlusion perception module to model the non-rigid motion of hair.
GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians
We present GaussianAvatar, an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video.
W-HMR: Monocular Human Mesh Recovery in World Space with Weak-Supervised Calibration
Our contributions include a novel weak-supervised camera calibration technique, an effective orientation correction module, and a decoupling strategy that significantly improves the generalizability and accuracy of human motion recovery in both camera and world coordinates.
Ranked #1 on
3D Human Pose Estimation
on SPEC-MTP
(using extra training data)
HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation
The main idea is to enhance the model's 2D perception of 3D geometry by learning a normal-adapted diffusion model and a normal-aligned diffusion model.
HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field
The problem of modeling an animatable 3D human head avatar under light-weight setups is of significant importance but has not been well solved.
Leveraging Intrinsic Properties for Non-Rigid Garment Alignment
To achieve wrinkle-level as well as texture-level alignment, we present a novel coarse-to-fine two-stage method that leverages intrinsic manifold properties with two neural deformation fields, in the 3D space and the intrinsic space, respectively.
CaPhy: Capturing Physical Properties for Animatable Human Avatars
In contrast to previous work on 3D avatar reconstruction, our method is able to generalize to novel poses with realistic dynamic cloth deformations.
ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning
For more accurate and physically plausible predictions in world space, our network is designed to learn human motions from a human-centric perspective, which enables the understanding of the same motion captured with different camera trajectories.
Ranked #231 on
3D Human Pose Estimation
on Human3.6M
Control4D: Efficient 4D Portrait Editing with Text
We introduce Control4D, an innovative framework for editing dynamic 4D portraits using text instructions.
Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images
In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space and leverage diffused contact probabilities to construct the implicit neural representation for the manipulated object.
AvatarReX: Real-time Expressive Full-body Avatars
We present AvatarReX, a new method for learning NeRF-based full-body avatars from video data.
LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar
Existing approaches to animatable NeRF-based head avatars are either built upon face templates or use the expression coefficients of templates as the driving signal.
StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video
Results and experiments demonstrate the superiority of our method in terms of image quality, full portrait video generation, and real-time re-animation compared to existing facial reenactment methods.
CloSET: Modeling Clothed Humans on Continuous Surface with Explicit Template Decomposition
In this way, the clothing deformations are disentangled such that the pose-dependent wrinkles can be better learned and applied to unseen poses.
Narrator: Towards Natural Control of Human-Scene Interaction Generation via Relationship Reasoning
Also, we model global and local spatial relationships in a 3D scene and a textual description respectively based on the scene graph, and introduce a partlevel action mechanism to represent interactions as atomic body part states.
Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery
The key components of a typical regressor lie in the feature extraction of input views and the fusion of multi-view features.
Ranked #2 on
Multi-view 3D Human Pose Estimation
on MPI-INF-3DHP
Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering
The key of our solution is an efficient 4D tensor decomposition method so that the dynamic scene can be directly represented as a 4D spatio-temporal tensor.
AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels
AvatarMAV is the first to model both the canonical appearance and the decoupled expression motion by neural voxels for head avatar.
Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering
The key of our solution is an efficient 4D tensor decomposition method so that the dynamic scene can be directly represented as a 4D spatio-temporal tensor.
Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars
We propose a novel 3D GAN framework for unsupervised learning of generative, high-quality and 3D-consistent facial avatars from unstructured 2D images.
DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras
At its core is a novel diffusion-based stereo module, which introduces diffusion models, a type of powerful generative models, into the iterative stereo matching network.
Learning Implicit Templates for Point-Based Clothed Human Modeling
We present FITE, a First-Implicit-Then-Explicit framework for modeling human avatars in clothing.
PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images
To address these issues, we propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop in our regression network for well-aligned human mesh recovery and extend it as PyMAF-X for the recovery of expressive full-body models.
Ranked #6 on
3D Human Pose Estimation
on AGORA
(using extra training data)
AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture
Then given a monocular RGB video of this subject, our method integrates information from both the image observation and the avatar prior, and accordingly recon-structs high-fidelity 3D textured models with dynamic details regardless of the visibility.
Structured Local Radiance Fields for Human Avatar Modeling
These local radiance fields not only leverage the flexibility of implicit representation in shape and appearance modeling, but also factorize cloth deformations into skeleton motions, node residual translations and the dynamic detail variations inside each individual radiance field.
Interacting Attention Graph for Single Image Two-Hand Reconstruction
To solve occlusion and interaction challenges of two-hand reconstruction, we introduce two novel attention based modules in each upsampling step of the original GCN.
Ranked #5 on
3D Interacting Hand Pose Estimation
on InterHand2.6M
3D Interacting Hand Pose Estimation
Vocal Bursts Valence Prediction
Recovering 3D Human Mesh from Monocular Images: A Survey
Since the release of statistical body models, 3D human mesh recovery has been drawing broader attention.
DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering
We introduce DoubleField, a novel framework combining the merits of both surface field and radiance field for high-fidelity human reconstruction and rendering.
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images.
Ranked #5 on
3D Human Pose Estimation
on AGORA
(using extra training data)
3D human pose and shape estimation
3D Human Reconstruction
+2
Evolving Search Space for Neural Architecture Search
The automation of neural architecture design has been a coveted alternative to human experts.
Rethinking Pseudo-LiDAR Representation
Based on this observation, we design an image based CNN detector named Patch-Net, which is more generalized and can be instantiated as pseudo-LiDAR based 3D detectors.
Cheaper Pre-training Lunch: An Efficient Paradigm for Object Detection
In this paper, we propose a general and efficient pre-training paradigm, Montage pre-training, for object detection.
Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics.
Ranked #4 on
3D Action Recognition
on Assembly101
Learning 3D Human Shape and Pose from Dense Body Parts
Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods.
Ranked #85 on
3D Human Pose Estimation
on 3DPW
(MPJPE metric)
3D human pose and shape estimation
3D Human Reconstruction
+3
Adversarial Learning Semantic Volume for 2D/3D Face Shape Regression in the Wild
Then, an end-to-end pipeline is designed to jointly regress the proposed volumetric representation and the coordinate vector.
Ranked #3 on
Face Alignment
on AFLW2000-3D
Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization
We decompose the prerequisite of warping into dense correspondence field estimation and facial texture map recovering, which are both well addressed by deep networks.
Android Malware Detection using Large-scale Network Representation Learning
With the growth of mobile devices and applications, the number of malicious software, or malware, is rapidly increasing in recent years, which calls for the development of advanced and effective malware detection approaches.
Cryptography and Security
Android Malware Detection based on Factorization Machine
As the popularity of Android smart phones has increased in recent years, so too has the number of malicious applications.
Cryptography and Security
Joint Voxel and Coordinate Regression for Accurate 3D Facial Landmark Localization
Then, a stacked hourglass network is adopted to estimate the volumetric representation from coarse to fine, followed by a 3D convolution network that takes the estimated volume as input and regresses 3D coordinates of the face shape.
Ranked #1 on
3D Facial Landmark Localization
on AFLW2000-3D
Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection
This Estimation-Correction-Tuning process perfectly combines the advantages of the global robustness of data-driven method (FCN), outlier correction capability of model-driven method (PDM) and non-parametric optimization of RLMS.
