Search Results for author:
Found 48 papers, 30 papers with code
EgoLife: Towards Egocentric Life Assistant
This effort resulted in the EgoLife Dataset, a comprehensive 300-hour egocentric, interpersonal, multiview, and multimodal daily life dataset with intensive annotation.
Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos
Existing methods either (1) assume sharp image inputs, failing to address the detail loss introduced by motion blur, or (2) mainly consider blur by camera movements, neglecting the human motion blur which is more common in animatable avatars.
SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation
To exclude the influence of algorithmic design, we base our experiments on two minimalist architectures: SMPLer-X, which consists of an intermediate step for hand and face localization, and SMPLest-X, an even simpler version that reduces the network to its bare essentials and highlights significant advances in the capture of articulated hands.
Disco4D: Disentangled 4D Human Generation and Animation from a Single Image
Different from existing methods, Disco4D distinctively disentangles clothings (with Gaussian models) from the human body (with SMPL-X model), significantly enhancing the generation details and flexibility.
MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers
Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement.
Large Motion Model for Unified Multi-Modal Motion Generation
In this work, we present Large Motion Model (LMM), a motion-centric, multi-modal framework that unifies mainstream motion generation tasks into a generalist model.
AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation
Expressive human pose and shape estimation (a. k. a.
Ranked #1 on
3D Multi-Person Mesh Recovery
on AGORA
WHAC: World-grounded Humans and Cameras
In this study, we aim to recover expressive parametric human models (i. e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera.
FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing
Notably, FineMoGen further enables zero-shot motion editing capabilities with the aid of modern large language models (LLM), which faithfully manipulates motion sequences with fine-grained instructions.
Ranked #6 on
Motion Synthesis
on KIT Motion-Language
Towards Robust and Expressive Whole-body Human Pose and Shape Estimation
Whole-body pose and shape estimation aims to jointly predict different behaviors (e. g., pose, hand gesture, facial expression) of the entire human body from a monocular image.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
Ranked #3 on
Motion Synthesis
on InterHuman
AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing
We propose AttriHuman-3D, an editable 3D human generation model, which address the aforementioned problems with attribute decomposition and indexing.
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
3D editing plays a crucial role in many areas such as gaming and virtual reality.
Story-to-Motion: Synthesizing Infinite and Controllable Character Animation from Long Text
Generating natural human motion from a story has the potential to transform the landscape of animation, gaming, and film industries.
SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation
1) For the data scaling, we perform a systematic investigation on 32 EHPS datasets, including a wide range of scenarios that a model trained on any single dataset cannot handle.
Ranked #2 on
3D Human Pose Estimation
on UBody
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
Recent strides in Text-to-3D techniques have been propelled by distilling knowledge from powerful large text-to-image diffusion models (LDMs).
Learning Dense UV Completion for Human Mesh Recovery
Our method utilizes a dense correspondence map to separate visible human features and completes human features on a structured UV map dense human with an attention-based feature completion module.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
Variational Relational Point Completion Network for Robust 3D Classification
Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details.
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
However, the performance on more diverse motions remains unsatisfactory.
Ranked #2 on
Motion Synthesis
on KIT Motion-Language
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets.
Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction
As it is hard to calibrate single-view RGB images in the wild, existing 3D human mesh reconstruction (3DHMR) methods either use a constant large focal length or estimate one based on the background environment context, which can not tackle the problem of the torso, limb, hand or face distortion caused by perspective camera projection when the camera is close to the human body.
Ranked #6 on
3D Human Pose Estimation
on 3DPW
BiBench: Benchmarking and Analyzing Network Binarization
Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width.
Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms
Experiments with 10 backbones, ranging from CNNs to transformers, show the knowledge learnt from a proximity task is readily transferable to human mesh recovery.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.
Ranked #25 on
Motion Synthesis
on KIT Motion-Language
Monocular 3D Object Reconstruction with GAN Inversion
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
PTTR: Relational 3D Point Cloud Object Tracking with Transformer
In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in the current search point cloud given a template point cloud.
Robust Partial-to-Partial Point Cloud Registration in a Full Range
\textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules.
Playing for 3D Human Recovery
Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.
MeshInversion: 3D textured mesh reconstruction with generative prior
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing
Generating an interpretable and compact representation of 3D shapes from point clouds is an important and challenging problem.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Delving Deep into the Generalization of Vision Transformers under Distribution Shifts
By comprehensively investigating these GE-ViTs and comparing with their corresponding CNN models, we observe: 1) For the enhanced model, larger ViTs still benefit more for the OOD generalization.
Unsupervised 3D Shape Completion through GAN Inversion
In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.
Variational Relational Point Completion Network
In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds.
Ranked #1 on
Point Cloud Completion
on Completion3D
REFINE: Prediction Fusion Network for Panoptic Segmentation
Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results.
Ranked #11 on
Panoptic Segmentation
on COCO test-dev
BiPointNet: Binary Neural Network for Point Clouds
To alleviate the resource constraint for real-time point cloud applications that run on edge devices, in this paper we present BiPointNet, the first model binarization approach for efficient deep learning on point clouds.
Balanced Activation for Long-tailed Visual Recognition
Deep classifiers have achieved great success in visual recognition.
Leveraging Localization for Multi-camera Association
We present McAssoc, a deep learning approach to the as-sociation of detection bounding boxes in different views ofa multi-camera system.
MessyTable: Instance Association in Multiple Camera Views
We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views.
Leveraging Temporal Information for 3D Detection and Domain Adaptation
Ever since the prevalent use of the LiDARs in autonomous driving, tremendous improvements have been made to the learning on the point clouds.
3D Convolution on RGB-D Point Clouds for Accurate Model-free Object Pose Estimation
The conventional pose estimation of a 3D object usually requires the knowledge of the 3D model of the object.
