Search Results for author:
Found 40 papers, 10 papers with code
GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations
Real-time rendering of human head avatars is a cornerstone of many computer graphics applications, such as augmented reality, video games, and films, to name a few.
Lite2Relight: 3D-aware Single Image Portrait Relighting
Achieving photorealistic 3D view synthesis and relighting of human portraits is pivotal for advancing AR/VR applications.
Live2Diff: Live Stream Translation via Uni-directional Attention in Video Diffusion Models
State-of-the-art video diffusion models leverage bi-directional temporal attention to model the correlations between the current frame and all the surrounding (i. e. including future) frames, which hinders them from processing streaming videos.
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.
GVP: Generative Volumetric Primitives
Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control.
HQ3DAvatar: High Quality Controllable 3D Head Avatar
This paper presents a novel approach to building highly photorealistic digital head avatars.
An Implicit Parametric Morphable Dental Model
Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum.
gCoRF: Generative Compositional Radiance Fields
We start with a global generative model (GAN) and learn to decompose it into different semantic parts using supervision from 2D segmentation masks.
EventNeRF: Neural Radiance Fields from a Single Colour Event Camera
Asynchronously operating event cameras find many applications due to their high dynamic range, vanishingly low motion blur, low latency and low data bandwidth.
φ-SfT: Shape-from-Template with a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
f-SfT: Shape-From-Template With a Physics-Based Deformation Model
In contrast to previous works, this paper proposes a new SfT approach explaining 2D observations through physical simulations accounting for forces and material properties.
NeRF for Outdoor Scene Relighting
Photorealistic editing of outdoor scenes from photographs requires a profound understanding of the image formation process and an accurate estimation of the scene geometry, reflectance and illumination.
StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN
We demonstrate that it suffices to train our temporal architecture on only 10 minutes of footage of 1 subject for about 6 hours.
Egocentric Videoconferencing
Even holding a mobile phone camera in the front of the face while sitting for a long duration is not convenient.
Self-supervised Outdoor Scene Relighting
Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo.
Differentiable Event Stream Simulator for Non-Rigid 3D Tracking
This paper introduces the first differentiable simulator of event streams, i. e., streams of asynchronous brightness change signals recorded by event cameras.
PhotoApp: Photorealistic Appearance Editing of Head Portraits
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
Learning Speech-driven 3D Conversational Gestures from Video
We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input.
Ranked #5 on
3D Face Animation
on BEAT2
High-Fidelity Neural Human Motion Transfer from Monocular Video
We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments.
EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream
Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams.
i3DMM: Deep Implicit 3D Morphable Model of Human Heads
Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair.
Learning Complete 3D Morphable Face Models from Images and Videos
Our network design and loss functions ensure a disentangled parameterization of not only identity and albedo, but also, for the first time, an expression basis.
PIE: Portrait Image Embedding for Semantic Control
We present the first approach for embedding real portrait images in the latent space of StyleGAN, which allows for intuitive editing of the head pose, facial expression, and scene illumination in the image.
User-assisted Video Reflection Removal
We show that user-assistance significantly improves the layer separation results.
Monocular Reconstruction of Neural Face Reflectance Fields
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
VideoForensicsHQ: Detecting High-quality Manipulated Face Videos
We introduce a new benchmark dataset for face video forgery detection, of unprecedented quality.
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination.
Neural Voice Puppetry: Audio-driven Facial Reenactment
Neural Voice Puppetry has a variety of use-cases, including audio-driven video avatars, video dubbing, and text-driven video synthesis of a talking head.
Neural Style-Preserving Visual Dubbing
We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages.
XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera
The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy.
Ranked #7 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Multi-Person Human Pose Estimation
3D Multi-Person Pose Estimation
+1
EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment
Our lightweight setup allows operations in uncontrolled environments, and lends itself to telepresence applications such as video-conferencing from dynamic environments.
FML: Face Model Learning from Videos
In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces.
A Dataset of Flash and Ambient Illumination Pairs from the Crowd
We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.
On Learning Associations of Faces and Voices
We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.
Learning-based Video Motion Magnification
We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.
Video Reflection Removal Through Spatio-Temporal Optimization
The novelty of our work is in our optimization formulation as well as the motion initialization strategy.
Efficient and Scalable View Generation from a Single Image using Fully Convolutional Networks
We present two FCN architectures for SIVG.
Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks
Since current datasets are not large enough to train an accurate SBD CNN, we present a new dataset containing more than 3. 5 million frames of sharp and gradual transitions.
Video Magnification in Presence of Large Motions
Video magnification reveals subtle variations that would be otherwise invisible to the naked eye.
Retrieval in Long Surveillance Videos using User Described Motion and Object Attributes
We present a content-based retrieval method for long surveillance videos both for wide-area (Airborne) as well as near-field imagery (CCTV).
