Search Results for author:
Found 43 papers, 16 papers with code
SqueezeMe: Efficient Gaussian Avatars for VR
To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space, and we distill the decoder to a single neural network layer.
URAvatar: Universal Relightable Gaussian Codec Avatars
Unlike existing approaches that estimate parametric reflectance parameters via inverse rendering, our approach directly models learnable radiance transfer that incorporates global light transport in an efficient manner for real-time rendering.
Agent-to-Sim: Learning Interactive Behavior Models from Casual Longitudinal Videos
We present Agent-to-Sim (ATS), a framework for learning interactive behavior models of 3D agents from casual longitudinal video collections.
Manifold Sampling for Differentiable Uncertainty in Radiance Fields
Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes.
3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion
The increasing demand for high-quality 3D assets across various industries necessitates efficient and automated 3D content creation.
Sapiens: Foundation for Human Vision Models
We present Sapiens, a family of models for four fundamental human-centric vision tasks -- 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction.
Ranked #1 on
Keypoint Detection
on MS COCO
(Validation AP metric, using extra
training data)
Expressive Whole-Body 3D Gaussian Avatar
Our hybrid representation treats each 3D Gaussian as a vertex on the surface with pre-defined connectivity information (i. e., triangle faces) between them following the mesh topology of SMPL-X.
Bridging the Gap: Studio-like Avatar Creation from a Monocular Phone Capture
In this paper, we propose a method that bridges this gap by generating studio-like illuminated texture maps from short, monocular phone captures.
InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion
Sampling from our model yields plausible and diverse two-hand shapes in close interaction with or without an object.
GALA: Generating Animatable Layered Assets from a Single Scan
Through a series of decomposition steps, we obtain multiple layers of 3D assets in a shared canonical space normalized in terms of poses and human shapes, hence supporting effortless composition to novel identities and reanimation with novel poses.
URHand: Universal Relightable Hands
To simplify the personalization process while retaining photorealism, we build a powerful universal relightable prior based on neural relighting from multi-view images of hands captured in a light stage with hundreds of identities.
Relightable Gaussian Codec Avatars
The fidelity of relighting is bounded by both geometry and appearance representations.
Single-Image 3D Human Digitization with Shape-Guided Diffusion
We present an approach to generate a 360-degree view of a person with a consistent, high-resolution appearance from a single input image.
Drivable 3D Gaussian Avatars
We present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats.
Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration
Registering clothes from 4D scans with vertex-accurate correspondence is challenging, yet important for dynamic appearance modeling and physics parameter estimation from real-world data.
Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering
We further conduct an extensive comparative study of different priors on illumination used in previous work on inverse rendering.
Neural Relighting with Subsurface Scattering by Learning the Radiance Transfer Gradient
Reconstructing and relighting objects and scenes under varying lighting conditions is challenging: existing neural rendering methods often cannot handle the complex interactions between materials and light.
NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objects
Our compositional model is interaction-aware, meaning the spatial relationship between humans and objects, and the mutual shape change by physical contact is fully incorporated.
MEGANE: Morphable Eyeglass and Avatar Network
However, modeling the geometric and appearance interactions of glasses and the face of virtual representations of humans is challenging.
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
To achieve generalization, we condition the student model with physics-inspired illumination features such as visibility, diffuse shading, and specular reflections computed on a coarse proxy geometry, maintaining a small computational overhead.
Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images
Furthermore, we introduce a novel neural rendering framework based on rasterization of the learned hair strands.
Drivable Volumetric Avatars using Texel-Aligned Features
To circumvent this, we propose a novel volumetric avatar representation by extending mixtures of volumetric primitives to articulated objects.
Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing
The key idea is to introduce a neural clothing appearance model that operates on top of explicit geometry: at training time we use high-fidelity tracking, whereas at animation time we rely on physically simulated geometry.
KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
In this work, we investigate common issues with existing spatial encodings and propose a simple yet highly effective approach to modeling high-fidelity volumetric humans from sparse views.
Ranked #2 on
Generalizable Novel View Synthesis
on ZJU-MoCap
COAP: Compositional Articulated Occupancy of People
We present a novel neural implicit representation for articulated human bodies.
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
In this work, for the first time, we enable autoregressive modeling of implicit avatars.
Neural Fields in Visual Computing and Beyond
Recent advances in machine learning have created increasing interest in solving visual computing problems using a class of coordinate-based neural networks that parametrize physical properties of scenes or objects across space and time.
ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
We present ARCH++, an image-based method to reconstruct 3D avatars with arbitrary clothing styles.
Ranked #1 on
3D Object Reconstruction From A Single Image
on RenderPeople
(using extra training data)
3D Object Reconstruction From A Single Image
Image-to-Image Translation
Pixel-Aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Ranked #5 on
Generalizable Novel View Synthesis
on ZJU-MoCap
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements
We demonstrate the efficacy of our surface representation by learning models of complex clothing from point clouds.
SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks
We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar.
Ranked #1 on
3D Human Reconstruction
on 4D-DRESS
PVA: Pixel-aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Monocular Real-Time Volumetric Performance Capture
We present the first approach to volumetric performance capture and novel-view rendering at real-time speed from monocular video, eliminating the need for expensive multi-view systems or cumbersome pre-acquisition of a personalized template model.
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Although current approaches have demonstrated the potential in real world settings, they still fail to produce reconstructions with the level of detail often present in the input images.
Ranked #1 on
3D Object Reconstruction From A Single Image
on BUFF
Learning to Infer Implicit Surfaces without 3D Supervision
The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output.
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object.
Ranked #1 on
3D Object Reconstruction
on RenderPeople
SiCloPe: Silhouette-Based Clothed People
The synthesized silhouettes which are the most consistent with the input segmentation are fed into a deep visual hull algorithm for robust 3D shape prediction.
Mesoscopic Facial Geometry Inference Using Deep Neural Networks
We present a learning-based approach for synthesizing facial geometry at medium and fine scales from diffusely-lit facial texture maps.
Realistic Dynamic Facial Textures From a Single Image Using GANs
By retargeting the PCA expression geometry from the source, as well as using the newly inferred texture, we can both animate the face and perform video face replacement on the source video using the target appearance.
Learning Dense Facial Correspondences in Unconstrained Images
To train such a network, we generate a massive dataset of synthetic faces with dense labels using renderings of a morphable face model with variations in pose, expressions, lighting, and occlusions.
Photorealistic Facial Texture Inference Using Deep Neural Networks
We present a data-driven inference method that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild.
Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks
We present a real-time deep learning framework for video-based facial performance capture -- the dense 3D tracking of an actor's face given a monocular video.
Real-Time Facial Segmentation and Performance Capture from RGB Input
We adopt a state-of-the-art regression-based facial tracking framework with segmented face images as training, and demonstrate accurate and uninterrupted facial performance capture in the presence of extreme occlusion and even side views.
