Search Results for author:
Found 37 papers, 7 papers with code
SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces
Our method generalizes to diverse real photographs and produces realistic illumination effects, including specular highlights and cast shadows, while preserving the subject's identity.
Free-viewpoint Human Animation with Pose-correlated Reference Selection
Diffusion-based human animation aims to animate a human character based on a source human image as well as driving signals such as a sequence of poses.
FaceLift: Single Image to 3D Head with View Generation and GS-LRM
Our pipeline begins by employing a multi-view latent diffusion model that generates consistent side and back views of the head from a single facial input.
MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data
We propose scaling up 3D scene reconstruction by training with synthesized data.
Generative Portrait Shadow Removal
For robust and natural shadow removal, we propose to train the diffusion model with a compositional repurposing framework: a pre-trained text-guided image generation model is first fine-tuned to harmonize the lighting and color of the foreground with a background scene by using a background harmonization dataset; and then the model is further fine-tuned to generate a shadow-free portrait image via a shadow-paired dataset.
Learning Frame-Wise Emotion Intensity for Audio-Driven Talking-Head Generation
The training signals for this predictor are obtained through our emotion-agnostic intensity pseudo-labeling method without the need of frame-wise intensity labeling.
COMPOSE: Comprehensive Portrait Shadow Editing
Existing portrait relighting methods struggle with precise control over facial shadows, particularly when faced with challenges such as handling hard shadows from directional light sources or adjusting shadows while remaining in harmony with existing lighting conditions.
Perm: A Parametric Representation for Multi-Style 3D Hair Modeling
We present Perm, a learned parametric representation of human 3D hair designed to facilitate various hair-related applications.
LRM-Zero: Training Large Reconstruction Models with Synthesized Data
We demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse, can achieve high visual quality in the reconstruction of real-world objects, competitive with models trained on Objaverse.
Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image
At the core of portrait photography is the search for ideal lighting and viewpoint.
Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos
Using a set of control signals, such as head pose and expressions, we transform them to the 3D space with learned deformations to generate the desired rendering.
Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
To this end, we introduce Carve3D, an improved RLFT algorithm coupled with a novel Multi-view Reconstruction Consistency (MRC) metric, to enhance the consistency of multi-view diffusion models.
Relightful Harmonization: Lighting-aware Portrait Background Replacement
Portrait harmonization aims to composite a subject into a new background, adjusting its lighting and color to ensure harmony with the background scene.
Controllable Dynamic Appearance for Neural 3D Portraits
The surface normals prediction is guided using 3DMM normals that act as a coarse prior for the normals of the human head, where direct prediction of normals is hard due to rigid and non-rigid deformations induced by head-pose and facial expression changes.
Consistent Multimodal Generation via A Unified GAN Framework
We investigate how to generate multimodal image outputs, such as RGB, depth, and surface normals, with a single generative model.
LightPainter: Interactive Portrait Relighting with Freehand Scribble
Recent portrait relighting methods have achieved realistic results of portrait lighting effects given a desired lighting representation such as an environment map.
In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing
3D-aware GANs offer new capabilities for view synthesis while preserving the editing functionalities of their 2D counterparts.
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis.
3D-FM GAN: Towards 3D-Controllable Face Manipulation
While concatenating GAN inversion and a 3D-aware, noise-to-image GAN is a straight-forward solution, it is inefficient and may lead to noticeable drop in editing quality.
RigNeRF: Fully Controllable Neural 3D Portraits
In this work, we propose RigNeRF, a system that goes beyond just novel view synthesis and enables full control of head pose and facial expressions learned from a single portrait video.
Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i. e., there exists a number of cloth geometric configurations given a pose depending on the way it has moved.
Point-NeRF: Point-based Neural Radiance Fields
Point-NeRF combines the advantages of these two approaches by using neural 3D point clouds, with associated neural features, to model a radiance field.
FLAME-in-NeRF: Neural control of Radiance Fields for Free View Face Animation
In this work, we design a system that enables 1) novel view synthesis for portrait video, of both the human subject and the scene they are in and 2) explicit control of the facial expressions through a low-dimensional expression representation.
Learning Surface Parameterization for Document Image Unwarping
We also demonstrate the usefulness of our system by applying it to document texture editing.
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
We present an algorithm for re-rendering a person from a single image under arbitrary poses.
FLAME-in-NeRF : Neural control of Radiance Fields for Free View Face Animation
In this work, we design a system that enables both novel view synthesis for portrait video, including the human subject and the scene background, and explicit control of the facial expressions through a low-dimensional expression representation.
Single-image Full-body Human Relighting
We present a single-image data-driven method to automatically relight images with full-body humans in them.
Content-Aware GAN Compression
We then propose a novel content-aware method to guide the processes of both pruning and distillation.
Learning Clusterable Visual Features for Zero-Shot Recognition
However, some testing data are considered "hard" as they lie close to the decision boundaries and are prone to misclassification, leading to performance degradation for ZSL.
Self-supervised Deformation Modeling for Facial Expression Editing
In the "motion-editing" step, we explicitly model facial movement through image deformation, warping the image into the desired expression.
Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion
In this work we introduce Lifting Autoencoders, a generative 3D surface-based model of object categories.
Latent Space Optimal Transport for Generative Models
Experimental results on an eight-Gaussian dataset show that the proposed OT can handle multi-cluster distributions.
Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
In this work we introduce Deforming Autoencoders, a generative model for images that disentangles shape from appearance in an unsupervised manner.
Ranked #9 on
Unsupervised Facial Landmark Detection
on MAFL
DocUNet: Document Image Unwarping via a Stacked U-Net
The network is trained on this dataset with various data augmentations to improve its generalization ability.
Ranked #4 on
SSIM
on DocUNet
(using extra training data)
An Adversarial Neuro-Tensorial Approach For Learning Disentangled Representations
Several factors contribute to the appearance of an object in a visual scene, including pose, illumination, and deformation, among others.
Improving Heterogeneous Face Recognition with Conditional Adversarial Networks
Heterogeneous face recognition between color image and depth image is a much desired capacity for real world applications where shape information is looked upon as merely involved in gallery.
Neural Face Editing with Intrinsic Image Disentangling
Traditional face editing methods often require a number of sophisticated and task specific algorithms to be applied one after the other --- a process that is tedious, fragile, and computationally intensive.
