Search Results for author:
Found 35 papers, 21 papers with code
GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
2) With the help of the pretrained human prior models, the Geometry Initialization-&-Sculpting pipeline is leveraged to recover high-quality 3D human geometry given a single image.
GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D Generation
While 3D content generation has advanced significantly, existing methods still face challenges with input formats, latent space design, and output representations.
EgoLM: Multi-Modal Language Model of Egocentric Motions
Multi-modal sensor inputs are encoded and projected to the joint latent space of language models, and used to prompt motion generation or text generation for egomotion tracking or understanding, respectively.
HMD$^2$: Environment-aware Motion Generation from Single Egocentric Head-Mounted Device
This paper investigates the online generation of realistic full-body human motion using a single head-mounted device with an outward-facing color camera and the ability to perform visual SLAM.
3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion
The increasing demand for high-quality 3D assets across various industries necessitates efficient and automated 3D content creation.
Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild
To the best of our knowledge, Nymeria dataset is the world's largest collection of human motion in the wild; first of its kind to provide synchronized and localized multi-device multimodal egocentric data; and the world's largest motion-language dataset.
GaussianCity: Generative Gaussian Splatting for Unbounded 3D City Generation
Recently 3D Gaussian Splatting (3D-GS) has emerged as a highly efficient alternative for object-level 3D generation.
Ranked #1 on
Scene Generation
on KITTI
4D Panoptic Scene Graph Generation
To facilitate research in this new area, we build a richly annotated PSG-4D dataset consisting of 3K RGB-D videos with a total of 1M frames, each of which is labeled with 4D panoptic segmentation masks as well as fine-grained, dynamic scene graphs.
DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation
Therefore, we introduce a diffusion-based feed-forward framework to address these challenges with a single model.
FashionEngine: Interactive 3D Human Generation and Editing via Multimodal Controls
FashionEngine automates the 3D human production with three key components: 1) A pre-trained 3D human diffusion model that learns to model 3D humans in a semantic UV latent space from 2D image training data, which provides strong priors for diverse generation and editing tasks.
StructLDM: Structured Latent Diffusion for 3D Human Generation
2) A structured 3D-aware auto-decoder that factorizes the global latent space into several semantic body parts parameterized by a set of conditional structured local NeRFs anchored to the body template, which embeds the properties learned from the 2D training data and can be decoded to render view-consistent humans under different poses and clothing styles.
SurMo: Surface-based 4D Motion Modeling for Dynamic Human Rendering
2) Physical motion decoding that is designed to encourage physical motion learning by decoding the motion triplane features at timestep t to predict both spatial derivatives and temporal derivatives at the next timestep t+1 in the training stage.
Large Motion Model for Unified Multi-Modal Motion Generation
In this work, we present Large Motion Model (LMM), a motion-centric, multi-modal framework that unifies mainstream motion generation tasks into a generalist model.
LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field.
3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping.
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
We present PrimDiffusion, the first diffusion-based framework for 3D human generation.
Large-Vocabulary 3D Diffusion Model with Transformer
To this end, we propose a novel triplane-based 3D-aware Diffusion model with TransFormer, DiffTF, for handling challenges via three aspects.
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
CityDreamer: Compositional Generative Model of Unbounded 3D Cities
3D city generation is a desirable yet challenging task, since humans are more sensitive to structural distortions in urban environments.
Ranked #2 on
Scene Generation
on GoogleEarth
(KID metric)
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
In this work, we propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process.
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
However, the performance on more diverse motions remains unsatisfactory.
Ranked #1 on
Motion Synthesis
on KIT Motion-Language
SHERF: Generalizable Human NeRF from a Single Image
To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.
DeformToon3D: Deformable Neural Radiance Fields for 3D Toonification
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
EVA3D: Compositional 3D Human Generation from 2D Image Collections
At the core of EVA3D is a compositional human NeRF representation, which divides the human body into local parts.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.
Ranked #21 on
Motion Synthesis
on KIT Motion-Language
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
Ranked #6 on
4D Panoptic Segmentation
on SemanticKITTI
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #2 on
Panoptic Segmentation
on SemanticKITTI
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited.
Ranked #3 on
3D Semantic Segmentation
on ScribbleKITTI
LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts
However, due to the irregularity and sparsity in sampled point clouds, it is hard to encode the fine-grained geometry of local regions and their spatial relationships when only using the fixed-size filters and individual local feature integration, which limit the ability to learn discriminative features.
