Search Results for author:
Found 42 papers, 17 papers with code
Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar
Rendering photorealistic and dynamically moving human heads is crucial for ensuring a pleasant and immersive experience in AR/VR and video conferencing applications.
Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry
Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing.
Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors
Specifically, we bridge the latent space of Get3DHuman with that of StyleGAN-Human via a specially-designed prior network, where the input latent code is mapped to the shape and texture feature volumes spanned by the pixel-aligned 3D reconstructor.
Audio2Gestures: Generating Diverse Gestures from Audio
Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline.
Learning Audio-Driven Viseme Dynamics for 3D Face Animation
We show that the predicted viseme curves can be applied to different viseme-rigged characters to yield various personalized animations with realistic and natural facial motions.
CARD: Semantic Segmentation with Efficient Class-Aware Regularized Decoder
Extensive experiments and ablation studies conducted on multiple benchmark datasets demonstrate that the proposed CAR can boost the accuracy of all baseline models by up to 2. 23% mIOU with superior generalization ability.
FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction
Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image inputs.
Ranked #3 on
3D Face Reconstruction
on REALY
Smooth image-to-image translations with latent space interpolations
Multi-domain image-to-image (I2I) translations can transform a source image according to the style of a target domain.
NEURAL MARIONETTE: A Transformer-based Multi-action Human Motion Synthesis System
Along with the novel system, we also present a new dataset dedicated to the multi-action motion synthesis task, which contains both action tags and their contextual information.
Learning to Construct 3D Building Wireframes from 3D Line Clouds
The network takes a line cloud as input , i. e., a nonstructural and unordered set of 3D line segments extracted from multi-view images, and outputs a 3D wireframe of the underlying building, which consists of a sparse set of 3D junctions connected by line segments.
PIFu for the Real World: A Self-supervised Framework to Reconstruct Dressed Human from Single-view Images
On synthetic data, our Intersection-Over-Union (IoU) achieves to 93. 5%, 18% higher compared with PIFuHD.
Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds
To guide the network quickly fit the coarse shape, we propose to utilize the signed supervision in regions that are obviously outside the object and can be easily determined, resulting in our semi-signed supervision.
REALY: Rethinking the Evaluation of 3D Face Reconstruction
The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan.
CAR: Class-aware Regularizations for Semantic Segmentation
Recent segmentation methods, such as OCR and CPNet, utilizing "class level" information in addition to pixel features, have achieved notable success for boosting the accuracy of existing network modules.
Ranked #8 on
Semantic Segmentation
on PASCAL Context
PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis
We present PVSeRF, a learning framework that reconstructs neural radiance fields from single-view RGB images, for novel view synthesis.
Consistent 3D Hand Reconstruction in Video via self-supervised Learning
We present a method for reconstructing accurate and consistent 3D hands from a monocular video.
NeRFReN: Neural Radiance Fields with Reflections
Specifically, we propose to split a scene into transmitted and reflected components, and model the two components with separate neural radiance fields.
ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation
The ISF manipulates the semantics of an input latent code to make the image generated from it lying in the desired visual domain.
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
In order to overcome this problem, we propose a novel conditional variational autoencoder (VAE) that explicitly models one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code.
Ranked #3 on
Gesture Generation
on BEAT
UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
Animatable Neural Radiance Fields from Monocular RGB Videos
We present animatable neural radiance fields (animatable NeRF) for detailed human avatar creation from monocular videos.
Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
In this paper, we propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which: 1) Both intra- and inter-domain interpolations correspond to gradual changes in the generated images and 2) The content of the source image is better preserved during the translation.
Model-based 3D Hand Reconstruction via Self-Supervised Learning
For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations.
High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies
We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera.
Self-supervised Video Representation Learning by Uncovering Spatio-temporal Statistics
Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc.
Task-agnostic Temporally Consistent Facial Video Editing
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusion
In this work, we propose to consider hand and object jointly in feature space and explore the reciprocity of the two branches.
Laplacian Denoising Autoencoder
This can be naturally generalized to span multiple scales with a Laplacian pyramid representation of the input data.
Self-supervised Learning of Detailed 3D Face Reconstruction
The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage.
MHP-VOS: Multiple Hypotheses Propagation for Video Object Segmentation
Extensive experiments on challenging datasets demonstrate the effectiveness of the proposed method, especially in the case of object missing.
MVF-Net: Multi-View 3D Face Morphable Model Regression
The main ingredient of the view alignment loss is a differentiable dense optical flow estimator that can backpropagate the alignment errors between an input view and a synthetic rendering from another input view, which is projected to the target view through the 3D shape to be inferred.
Self-supervised Spatio-temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics
We conduct extensive experiments with C3D to validate the effectiveness of our proposed approach.
Ranked #47 on
Self-Supervised Action Recognition
on HMDB51
Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement
We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image.
Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry
Our approach is able to handle the rolling-shutter effects and imperfect sensor synchronization in a unified way.
Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks
The proposed network is composed of three deep convolutional neural networks (CNNs) and a recurrent neural network (RNN).
Ranked #10 on
Deblurring
on RealBlur-R (trained on GoPro)
(SSIM (sRGB) metric)
VITAL: VIsual Tracking via Adversarial Learning
To augment positive samples, we use a generative network to randomly generate masks, which are applied to adaptively dropout input features to capture a variety of appearance changes.
CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRF
With temporal dependencies established by optical flow, the resulting MRF model combines both spatial and temporal cues for tackling video object segmentation.
Ranked #3 on
Semi-Supervised Video Object Segmentation
on YouTube
Stylizing Face Images via Multiple Exemplars
We address the problem of transferring the style of a headshot photo to face images.
Fast Preprocessing for Robust Face Sketch Synthesis
Exemplar-based face sketch synthesis methods usually meet the challenging problem that input photos are captured in different lighting conditions from training photos.
Learning to Hallucinate Face Images via Component Generation and Enhancement
We propose a two-stage method for face hallucination.
Robust Piecewise-Constant Smoothing: M-Smoother Revisited
In addition, high-quality piecewise-constant smoothing can be achieved via a number of bilateral filtering or guided filtering integrated in the proposed framework.
Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow
We present a fast optical flow algorithm that can handle large displacement motions.
