Search Results for author:
Found 32 papers, 14 papers with code
Relightable Full-Body Gaussian Codec Avatars
This allows us to learn diffuse radiance transfer in a local coordinate frame, which disentangles the local radiance transfer from the articulation of the body.
SqueezeMe: Efficient Gaussian Avatars for VR
To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space, and we distill the decoder to a single neural network layer.
URAvatar: Universal Relightable Gaussian Codec Avatars
Unlike existing approaches that estimate parametric reflectance parameters via inverse rendering, our approach directly models learnable radiance transfer that incorporates global light transport in an efficient manner for real-time rendering.
Universal Facial Encoding of Codec Avatars from VR Headsets
The oblique and incomplete views of the face, variability in the donning of headsets, and illumination variation due to the environment are some of the unique challenges in generalization to unseen faces.
Rasterized Edge Gradients: Handling Discontinuities Differentiably
Computing the gradients of a rendering process is paramount for diverse applications in computer vision and graphics.
URHand: Universal Relightable Hands
To simplify the personalization process while retaining photorealism, we build a powerful universal relightable prior based on neural relighting from multi-view images of hands captured in a light stage with hundreds of identities.
Relightable Gaussian Codec Avatars
The fidelity of relighting is bounded by both geometry and appearance representations.
MEGANE: Morphable Eyeglass and Avatar Network
However, modeling the geometric and appearance interactions of glasses and the face of virtual representations of humans is challenging.
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
To achieve generalization, we condition the student model with physics-inspired illumination features such as visibility, diffuse shading, and specular reflections computed on a coarse proxy geometry, maintaining a small computational overhead.
Multiface: A Dataset for Neural Face Rendering
Along with the release of the dataset, we conduct ablation studies on the influence of different model architectures toward the model's interpolation capacity of novel viewpoint and expressions.
Drivable Volumetric Avatars using Texel-Aligned Features
To circumvent this, we propose a novel volumetric avatar representation by extending mixtures of volumetric primitives to articulated objects.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
Pixel-Aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Ranked #5 on
Generalizable Novel View Synthesis
on ZJU-MoCap
Driving-Signal Aware Full-Body Avatars
The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation.
Pixel Codec Avatars
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical distances.
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Based on this refined kinematic pose, the policy learns to compute dynamics-based control (e. g., joint torques) of the character to advance the current-frame pose estimate to the pose estimate of the next frame.
Ranked #252 on
3D Human Pose Estimation
on Human3.6M
Mixture of Volumetric Primitives for Efficient Neural Rendering
Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications.
PVA: Pixel-aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Learning Compositional Radiance Fields of Dynamic Human Heads
In addition, we show that the learned dynamic radiance field can be used to synthesize novel unseen expressions based on a global animation code.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Although current approaches have demonstrated the potential in real world settings, they still fail to produce reconstructions with the level of detail often present in the input images.
Ranked #1 on
3D Object Reconstruction From A Single Image
on BUFF
Single-Network Whole-Body Pose Estimation
We present the first single-network approach for 2D~whole-body pose estimation, which entails simultaneous localization of body, face, hands, and feet keypoints.
Neural Volumes: Learning Dynamic Renderable Volumes from Images
Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion.
Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction
We present a new research task and a dataset to understand human social interactions via computational methods, to ultimately endow machines with the ability to encode and decode a broad channel of social signals humans use.
LBS Autoencoder: Self-supervised Fitting of Articulated Meshes to Point Clouds
As input, we take a sequence of point clouds to be registered as well as an artist-rigged mesh, i. e. a template mesh equipped with a linear-blend skinning (LBS) deformation space parameterized by a skeleton hierarchy.
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation
Ranked #4 on
Pose Estimation
on MPII Single Person
Deep Appearance Models for Face Rendering
At inference time, we condition the decoding network on the viewpoint of the camera in order to generate the appropriate texture for rendering.
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
We present a unified deformation model for the markerless capture of multiple scales of human movement, including facial expressions, body motion, and hand gestures.
Hand Keypoint Detection in Single Images using Multiview Bootstrapping
The method is used to train a hand keypoint detector for single images.
Panoptic Studio: A Massively Multiview System for Social Interaction Capture
The core challenges in capturing social interactions are: (1) occlusion is functional and frequent; (2) subtle motion needs to be measured over a space large enough to host a social group; (3) human appearance and configuration variation is immense; and (4) attaching markers to the body may prime the nature of interactions.
Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
We present an approach to efficiently detect the 2D pose of multiple people in an image.
Ranked #1 on
Multi-Person Pose Estimation
on COCO
Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction
This paper proposes photogeometric scene flow (PGSF) for high-quality dynamic 3D reconstruction.
