Search Results for author:
Found 26 papers, 8 papers with code
Modeling Facial Geometry Using Compositional VAEs
We propose a method for learning non-linear face geometry representations using deep generative models.
Deep Appearance Models for Face Rendering
At inference time, we condition the decoding network on the viewpoint of the camera in order to generate the appropriate texture for rendering.
Image Disentanglement and Uncooperative Re-Entanglement for High-Fidelity Image-to-Image Translation
Cross-domain image-to-image translation should satisfy two requirements: (1) preserve the information that is common to both domains, and (2) generate convincing images covering variations that appear in the target domain.
LBS Autoencoder: Self-supervised Fitting of Articulated Meshes to Point Clouds
As input, we take a sequence of point clouds to be registered as well as an artist-rigged mesh, i. e. a template mesh equipped with a linear-blend skinning (LBS) deformation space parameterized by a skeleton hierarchy.
Neural Volumes: Learning Dynamic Renderable Volumes from Images
Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion.
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Although current approaches have demonstrated the potential in real world settings, they still fail to produce reconstructions with the level of detail often present in the input images.
Ranked #1 on
3D Object Reconstruction From A Single Image
on BUFF
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels
Learning latent representations of registered meshes is useful for many 3D tasks.
Learning Compositional Radiance Fields of Dynamic Human Heads
In addition, we show that the learned dynamic radiance field can be used to synthesize novel unseen expressions based on a global animation code.
PVA: Pixel-aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Mixture of Volumetric Primitives for Efficient Neural Rendering
Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications.
High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
This paper addresses previous limitations by learning a deep learning lighting model, that in combination with a high-quality 3D face tracking algorithm, provides a method for subtle and robust facial motion transfer from a regular video to a 3D photo-realistic avatar.
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Based on this refined kinematic pose, the policy learns to compute dynamics-based control (e. g., joint torques) of the character to advance the current-frame pose estimate to the pose estimate of the next frame.
Ranked #229 on
3D Human Pose Estimation
on Human3.6M
Pixel Codec Avatars
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical distances.
Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality
Social presence, the feeling of being there with a real person, will fuel the next generation of communication systems driven by digital humans in virtual reality (VR).
Driving-Signal Aware Full-Body Avatars
The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation.
Pixel-Aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Ranked #5 on
Generalizable Novel View Synthesis
on ZJU-MoCap
LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space
To mitigate this asymmetry, we introduce a prior model that is conditioned on the runtime inputs and tie this prior space to the 3D face model via a normalizing flow in the latent space.
Drivable Volumetric Avatars using Texel-Aligned Features
To circumvent this, we propose a novel volumetric avatar representation by extending mixtures of volumetric primitives to articulated objects.
Multiface: A Dataset for Neural Face Rendering
Along with the release of the dataset, we conduct ablation studies on the influence of different model architectures toward the model's interpolation capacity of novel viewpoint and expressions.
NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation
The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality.
MEGANE: Morphable Eyeglass and Avatar Network
However, modeling the geometric and appearance interactions of glasses and the face of virtual representations of humans is challenging.
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
To achieve generalization, we condition the student model with physics-inspired illumination features such as visibility, diffuse shading, and specular reflections computed on a coarse proxy geometry, maintaining a small computational overhead.
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence.
A Local Appearance Model for Volumetric Capture of Diverse Hairstyle
In this paper, we present a novel method for creating high-fidelity avatars with diverse hairstyles.
Fast Registration of Photorealistic Avatars for VR Facial Animation
In this work, we first show that the domain gap between the avatar and headset-camera images is one of the primary sources of difficulty, where a transformer-based architecture achieves high accuracy on domain-consistent data, but degrades when the domain-gap is re-introduced.
