Search Results for author:
Found 64 papers, 26 papers with code
Synthetic Prior for Few-Shot Drivable Head Avatar Inversion
With few input images, SynShot fine-tunes the pretrained synthetic prior to bridge the domain gap, modeling a photorealistic head avatar that generalizes to novel expressions and viewpoints.
GaussianSpeech: Audio-Driven Gaussian Avatars
We introduce GaussianSpeech, a novel approach that synthesizes high-fidelity animation sequences of photo-realistic, personalized 3D human head avatars from spoken audio.
Joker: Conditional 3D Head Synthesis with Extreme Facial Expressions
We introduce Joker, a new method for the conditional synthesis of 3D human heads with extreme expressions.
Stable Video Portraits
Rapid advances in the field of generative AI and text-to-image methods in particular have transformed the way we interact with and perceive computer-generated imagery today.
Human Hair Reconstruction with Strand-Aligned 3D Gaussians
We introduce a new hair modeling method that uses a dual representation of classical hair strands and 3D Gaussians to produce accurate and realistic strand-based reconstructions from multi-view data.
Gaussian Eigen Models for Human Heads
Building on the success of mesh-based 3D morphable face models (3DMM), we define GEM as an ensemble of linear eigenbases for representing the head appearance of a specific subject.
Generating Human Interaction Motions in Scenes with Text Control
Our approach begins with pre-training a scene-agnostic text-to-motion diffusion model, emphasizing goal-reaching constraints on large-scale motion-capture datasets.
Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles
We present HAAR a new strand-based generative model for 3D human hairstyles.
Synthesizing Environment-Specific People in Photographs
We present ESP, a novel method for context-aware full-body generation, that enables photo-realistic synthesis and inpainting of people wearing clothing that is semantically appropriate for the scene depicted in an input photograph.
HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles
We present HAAR, a new strand-based generative model for 3D human hairstyles.
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models
We propose a new latent diffusion model for this task, operating in the expression space of neural parametric head models, to synthesize audio-driven realistic head sequences.
360° Volumetric Portrait Avatar
In contrast to this, we propose a template-based tracking of the torso, head and facial expressions which allows us to cover the appearance of a human subject from all sides.
DPHMs: Diffusion Parametric Head Models for Depth-based Tracking
We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences.
3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing
To enable stochasticity as well as motion editing, we propose a lightweight audio-conditioned diffusion model for 3D facial motion.
GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar
For controlling the model, we learn a mapping from 3DMM facial expression parameters to the latent space of the generative model.
Drivable 3D Gaussian Avatars
We present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats.
Text-Guided Generation and Editing of Compositional 3D Avatars
We argue that existing methods are limited because they employ a monolithic modeling approach, using a single representation for the head, face, hair, and accessories.
SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes
We use intermediate activations of the learned geometry model to condition our texture generator.
TADA! Text to Animatable Digital Avatars
We introduce TADA, a simple-yet-effective approach that takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures, that can be animated and rendered with traditional graphics pipelines.
TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
But how to effectively capture all visual attributes of an individual from a single image, which are sufficient to reconstruct unseen areas (e. g., the back view)?
CaPhy: Capturing Physical Properties for Animatable Human Avatars
In contrast to previous work on 3D avatar reconstruction, our method is able to generalize to novel poses with realistic dynamic cloth deformations.
DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis
We introduce a diffusion network to synthesize a collection of 3D indoor objects by denoising a set of unordered object attributes.
Imitator: Personalized Speech-driven 3D Facial Animation
To address this, we present Imitator, a speech-driven facial expression synthesis method, which learns identity-specific details from a short input video and produces novel facial expressions matching the identity-specific speaking style and facial idiosyncrasies of the target actor.
MIME: Human-Aware 3D Scene Generation
We propose MIME (Mining Interaction and Movement to infer 3D Environments), which is a generative model of indoor scenes that produces furniture layouts that are consistent with the human movement.
Ranked #2 on
Indoor Scene Synthesis
on PRO-teXt
2D Semantic Segmentation task 1 (8 classes)
3D Semantic Scene Completion
+2
High-Res Facial Appearance Capture from Polarized Smartphone Images
We propose a novel method for high-quality facial texture reconstruction from RGB images using a novel capturing routine based on a single smartphone which we equip with an inexpensive polarization foil.
ClipFace: Text-guided Editing of Textured 3D Morphable Models
Controllable editing and manipulation are given by language prompts to adapt texture and expression of the 3D morphable model.
DINER: Depth-aware Image-based NEural Radiance fields
We present Depth-aware Image-based NEural Radiance fields (DINER).
Instant Volumetric Head Avatars
In addition, it allows for the interactive rendering of novel poses and expressions.
Neural Shape Deformation Priors
To this end, we introduce transformer-based deformation networks that represent a shape deformation as a composition of local surface deformations.
Towards Metrical Reconstruction of Human Faces
To this end, we take advantage of a face recognition network pretrained on a large-scale 2D image dataset, which provides distinct features for different faces and is robust to expression, illumination, and camera changes.
Ranked #2 on
3D Face Reconstruction
on NoW Benchmark
Texturify: Generating Textures on 3D Shape Surfaces
Texture cues on 3D objects are key to compelling visual representations, with the possibility to create high visual fidelity with inherent spatial consistency across different views.
Human-Aware Object Placement for Visual Environment Reconstruction
In fact, we demonstrate that these human-scene interactions (HSIs) can be leveraged to improve the 3D reconstruction of a scene from a monocular RGB video.
Neural Head Avatars from Monocular RGB Videos
We present Neural Head Avatars, a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar that can be used for teleconferencing in AR/VR or other applications in the movie or games industry that rely on a digital human.
Advances in Neural Rendering
The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering.
Egocentric Videoconferencing
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
TransformerFusion: Monocular RGB Scene Reconstruction using Transformers
We introduce TransformerFusion, a transformer-based 3D scene reconstruction approach.
Neural RGB-D Surface Reconstruction
Obtaining high-quality 3D reconstructions of room-scale scenes is of paramount importance for upcoming applications in AR or VR.
Dynamic Surface Function Networks for Clothed Human Bodies
To this end, we explicitly model the surface of the person using a multi-layer perceptron (MLP) which is embedded into the canonical space of the SMPL body model.
NPMs: Neural Parametric Models for 3D Deformable Shapes
Crucially, once learned, our neural parametric models of shape and pose enable optimization over the learned spaces to fit to new observations, similar to the fitting of a traditional parametric model, e. g., SMPL.
RetrievalFuse: Neural 3D Scene Reconstruction with a Database
3D reconstruction of large scenes is a challenging problem due to the high-complexity nature of the solution space, in particular for generative neural networks.
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face.
ID-Reveal: Identity-aware DeepFake Video Detection
A major challenge in DeepFake forgery detection is that state-of-the-art algorithms are mostly trained to detect a specific fake method.
Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction
We introduce Neural Deformation Graphs for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.
Face2Face: Real-time Face Capture and Reenactment of RGB Videos
Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.
Learning Adaptive Sampling and Reconstruction for Volume Visualization
A central challenge in data visualization is to understand which data samples are required to generate an image of a data set in which the relevant information is encoded.
Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
Neural rendering techniques promise efficient photo-realistic image synthesis while at the same time providing rich control over scene parameters by learning the physical image formation process.
SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans
We present SPSG, a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations by learning to infer unobserved scene geometry and color in a self-supervised fashion.
Neural Non-Rigid Tracking
We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
Adversarial Texture Optimization from RGB-D Scans
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
Neural Voice Puppetry: Audio-driven Facial Reenactment
Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.
SpoC: Spoofing Camera Fingerprints
Given a GAN-generated image, we insert the traces of a specific camera model into it and deceive state-of-the-art detectors into believing the image was acquired by that model.
Deferred Neural Rendering: Image Synthesis using Neural Textures
Similar to traditional textures, neural textures are stored as maps on top of 3D mesh proxies; however, the high-dimensional feature maps contain significantly more information, which can be interpreted by our new deferred neural rendering pipeline.
FaceForensics++: Learning to Detect Manipulated Facial Images
In particular, the benchmark is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size.
Ranked #1 on
DeepFake Detection
on FaceForensics
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
We devise a learning-based forensic detector which adapts well to new domains, i. e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training.
DeepVoxels: Learning Persistent 3D Feature Embeddings
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
IGNOR: Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
Deep Video Portraits
In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target.
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze.
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Research on the detection of face manipulations has been seriously hampered by the lack of adequate datasets.
IMU2Face: Real-time Gesture-driven Facial Reenactment
We present IMU2Face, a gesture-driven facial reenactment system.
InverseFaceNet: Deep Monocular Inverse Face Rendering
In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus.
FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality
Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances.
