Search Results for author:
Found 42 papers, 15 papers with code
Taming Diffusion Probabilistic Models for Character Control
We present a novel character control framework that effectively utilizes motion diffusion probabilistic models to generate high-quality and diverse character animations, responding in real-time to a variety of dynamic user-supplied control signals.
Coverage Axis++: Efficient Inner Point Selection for 3D Shape Skeletonization
We introduce Coverage Axis++, a novel and efficient approach to 3D shape skeletonization.
On Optimal Sampling for Learning SDF Using MLPs Equipped with Positional Encoding
Sampling against this intrinsic frequency following the Nyquist-Sannon sampling theorem allows us to determine an appropriate training sampling rate.
DiffusionPhase: Motion Diffusion in Frequency Domain
In this study, we introduce a learning-based method for generating high-quality human motion sequences from text descriptions (e. g., ``A person walks forward").
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
We introduce Efficient Motion Diffusion Model (EMDM) for fast and high-quality human motion generation.
Generative Hierarchical Temporal Transformer for Hand Action Recognition and Motion Prediction
We present a novel framework that concurrently tackles hand action recognition and 3D future hand motion prediction.
StructRe: Rewriting for Structured Shape Modeling
Such a localized rewriting process enables probabilistic modeling of ambiguous structures and robust generalization across object categories.
TLControl: Trajectory and Language Control for Human Motion Synthesis
Controllable human motion synthesis is essential for applications in AR/VR, gaming, movies, and embodied AI.
Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models
The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions.
C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
We present C$\cdot$ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for physics-based characters.
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer
Learning the mapping between speech and 3D full-body gestures is difficult due to the stochastic nature of the problem and the lack of a rich cross-modal dataset that is needed for training.
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
In this paper, we present a novel diffusion model called that generates multiview-consistent images from a single-view image.
Motion In-Betweening with Phase Manifolds
This paper introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder.
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.
Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild
We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild.
F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
As it is hard to calibrate single-view RGB images in the wild, existing 3D human mesh reconstruction (3DHMR) methods either use a constant large focal length or estimate one based on the background environment context, which can not tackle the problem of the torso, limb, hand or face distortion caused by perspective camera projection when the camera is close to the human body.
Ranked #5 on
3D Human Pose Estimation
on 3DPW
Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians
The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique.
F2-NeRF: Fast Neural Radiance Field Training With Free Camera Trajectories
Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes.
PhaseMP: Robust 3D Pose Estimation via Phase-conditioned Human Motion Prior
We present a novel motion prior, called PhaseMP, modeling a probability distribution on pose transitions conditioned by a frequency domain feature extracted from a periodic autoencoder.
NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies
In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
We further demonstrate the generalizability of our method on hand mesh recovery.
Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos
Understanding dynamic hand motions and actions from egocentric RGB videos is a fundamental yet challenging task due to self-occlusion and ambiguity.
Progressively-connected Light Field Network for Efficient View Synthesis
ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses.
NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors
The key idea of NeuRIS is to integrate estimated normal of indoor scenes as a prior in a neural rendering framework for reconstructing large texture-less shapes and, importantly, to do this in an adaptive manner to also enable the reconstruction of irregular shapes with fine details.
Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions
We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.
SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images.
Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
In this paper, we present a generalizable model-free 6-DoF object pose estimator called Gen6D.
Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases
In this work we present a style modelling system that uses an animation synthesis network to model motion content based on local motion phases.
FaceFormer: Speech-Driven 3D Facial Animation with Transformers
Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data.
Ranked #1 on
3D Face Animation
on VOCASET
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation
The existing datasets are collected to cover as many different phonemes as possible instead of sentences, thus limiting the capability of the audio-based model to learn more diverse contexts.
DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation
Scalable 6D pose estimation for rigid objects from RGB images aims at handling multiple objects and generalizing to novel objects.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.
MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. While previous methods rely on either rigging or inverse kinematics (IK) to associate a consistent skeleton with temporally coherent joint rotations, our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
Local motion phases for learning multi-contact character movements
Training a bipedal character to play basketball and interact with objects, or a quadruped character to move in various locomotion modes, are difficult tasks due to the fast and complex contacts happening during the motion.
Learning Whole-body Motor Skills for Humanoids
This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i. e., ankle, hip, foot tilting, and stepping strategies.
A new method to evaluate the dynamic air gap thickness and garment sliding of virtual clothes during walking
However, the virtual clothing fit evaluation is still under-researched.
Personalized 3D mannequin reconstruction based on 3D scanning
A personalized mannequin is essential for apparel customization using CAD technologies.
3D textile reconstruction based on KinectFusion and synthesized texture
Purpose The purpose of this paper is to present a novel framework of reconstructing the 3D textile model with synthesized texture.
Scanning and animating characters dressed in multiple-layer garments
In this paper, we propose a novel scanning-based solution for modeling and animating characters wearing multiple layers of clothes.
