Search Results for author:
Found 24 papers, 8 papers with code
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence.
MEGANE: Morphable Eyeglass and Avatar Network
However, modeling the geometric and appearance interactions of glasses and the face of virtual representations of humans is challenging.
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
To achieve generalization, we condition the student model with physics-inspired illumination features such as visibility, diffuse shading, and specular reflections computed on a coarse proxy geometry, maintaining a small computational overhead.
NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation
The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality.
Multiface: A Dataset for Neural Face Rendering
Along with the release of the dataset, we conduct ablation studies on the influence of different model architectures toward the model's interpolation capacity of novel viewpoint and expressions.
Drivable Volumetric Avatars using Texel-Aligned Features
To circumvent this, we propose a novel volumetric avatar representation by extending mixtures of volumetric primitives to articulated objects.
LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space
To mitigate this asymmetry, we introduce a prior model that is conditioned on the runtime inputs and tie this prior space to the 3D face model via a normalizing flow in the latent space.
Pixel-Aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Ranked #4 on
Generalizable Novel View Synthesis
on ZJU-MoCap
Driving-Signal Aware Full-Body Avatars
The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation.
Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality
Social presence, the feeling of being there with a real person, will fuel the next generation of communication systems driven by digital humans in virtual reality (VR).
Pixel Codec Avatars
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical distances.
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Based on this refined kinematic pose, the policy learns to compute dynamics-based control (e. g., joint torques) of the character to advance the current-frame pose estimate to the pose estimate of the next frame.
Ranked #226 on
3D Human Pose Estimation
on Human3.6M
High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
This paper addresses previous limitations by learning a deep learning lighting model, that in combination with a high-quality 3D face tracking algorithm, provides a method for subtle and robust facial motion transfer from a regular video to a 3D photo-realistic avatar.
Mixture of Volumetric Primitives for Efficient Neural Rendering
Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications.
PVA: Pixel-aligned Volumetric Avatars
Volumetric models typically employ a global code to represent facial expressions, such that they can be driven by a small set of animation parameters.
Learning Compositional Radiance Fields of Dynamic Human Heads
In addition, we show that the learned dynamic radiance field can be used to synthesize novel unseen expressions based on a global animation code.
Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels
Learning latent representations of registered meshes is useful for many 3D tasks.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Although current approaches have demonstrated the potential in real world settings, they still fail to produce reconstructions with the level of detail often present in the input images.
Ranked #1 on
3D Object Reconstruction From A Single Image
on BUFF
Neural Volumes: Learning Dynamic Renderable Volumes from Images
Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion.
LBS Autoencoder: Self-supervised Fitting of Articulated Meshes to Point Clouds
As input, we take a sequence of point clouds to be registered as well as an artist-rigged mesh, i. e. a template mesh equipped with a linear-blend skinning (LBS) deformation space parameterized by a skeleton hierarchy.
Image Disentanglement and Uncooperative Re-Entanglement for High-Fidelity Image-to-Image Translation
Cross-domain image-to-image translation should satisfy two requirements: (1) preserve the information that is common to both domains, and (2) generate convincing images covering variations that appear in the target domain.
Deep Appearance Models for Face Rendering
At inference time, we condition the decoding network on the viewpoint of the camera in order to generate the appropriate texture for rendering.
Modeling Facial Geometry Using Compositional VAEs
We propose a method for learning non-linear face geometry representations using deep generative models.
