Search Results for author:
Found 39 papers, 13 papers with code
Neural Pixel Composition for 3D-4D View Synthesis From Multi-Views
We present Neural Pixel Composition (NPC), a novel approach for continuous 3D-4D view synthesis given only a discrete set of multi-view observations as input.
AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields
However, rendering images with this new paradigm is slow due to the fact that an accurate quadrature of the volume rendering equation requires a large number of samples for each ray.
Self-supervised Neural Articulated Shape and Appearance Models
In addition, our representation enables a large variety of applications, such as few-shot reconstruction, the generation of novel articulations, and novel view-synthesis.
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
In this work, for the first time, we enable autoregressive modeling of implicit avatars.
Learning Neural Light Fields With Ray-Space Embedding
Our method supports rendering with a single network evaluation per pixel for small baseline light fields and with only a few evaluations per pixel for light fields with larger baselines.
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video
We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e. g. a `bullet-time' video effect.
Learning Compositional Radiance Fields of Dynamic Human Heads
In addition, we show that the learned dynamic radiance field can be used to synthesize novel unseen expressions based on a global animation code.
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face.
Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction
We introduce Neural Deformation Graphs for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.
PIE: Portrait Image Embedding for Semantic Control
We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image.
PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations
At the level of patches, objects across different categories share similarities, which leads to more generalizable models.
Face2Face: Real-time Face Capture and Reenactment of RGB Videos
Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.
Neural Non-Rigid Tracking
We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination.
Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data
Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus.
Neural Style-Preserving Visual Dubbing
We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes.
Text-based Editing of Talking-head Video
To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material.
DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects
Mesh autoencoders are commonly used for dimensionality reduction, sampling and mesh modeling.
Deferred Neural Rendering: Image Synthesis using Neural Textures
Similar to traditional textures, neural textures are stored as maps on top of 3D mesh proxies; however, the high-dimensional feature maps contain significantly more information, which can be interpreted by our new deferred neural rendering pipeline.
Commodity RGB-D Sensors: Data Acquisition
Over the past ten years we have seen a democratization of range sensing technology.
FML: Face Model Learning from Videos
In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces.
DeepVoxels: Learning Persistent 3D Feature Embeddings
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis.
IGNOR: Image-guided Neural Object Rendering
Based on this 3D proxy, the appearance of a captured view can be warped into a new target view as in classical image-based rendering.
Deep Video Portraits
In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target.
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze.
IMU2Face: Real-time Gesture-driven Facial Reenactment
We present IMU2Face, a gesture-driven facial reenactment system.
Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz
To alleviate this problem, we present the first approach that jointly learns 1) a regressor for face shape, expression, reflectance and illumination on the basis of 2) a concurrently learned parametric face model.
MonoPerfCap: Human Performance Capture from Monocular Video
Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem.
InverseFaceNet: Deep Monocular Inverse Face Rendering
In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus.
MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image.
Real-time Halfway Domain Reconstruction of Motion and Geometry
Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.
Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
However, due to difficult occlusions, fast motions, and uniform hand appearance, jointly tracking hand and object pose is more challenging than tracking either of the two separately.
FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality
Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances.
Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging
Many graphics and vision problems can be expressed as non-linear least squares optimizations of objective functions over visual data, such as images and meshes.
BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration
Our approach estimates globally optimized (i. e., bundle adjusted) poses in real-time, supports robust tracking with recovery from gross tracking failures (i. e., relocalization), and re-estimates the 3D model in real-time to ensure global consistency; all within a single framework.
VolumeDeform: Real-time Volumetric Non-rigid Reconstruction
We cast finding the optimal deformation of space as a non-linear regularized variational optimization problem by enforcing local smoothness and proximity to the input constraints.
