Search Results for author: Linchao Bao

Found 42 papers, 17 papers with code

Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar

no code implementations11 Jul 2023 Cong Wang, Di Kang, Yan-Pei Cao, Linchao Bao, Ying Shan, Song-Hai Zhang

Rendering photorealistic and dynamically moving human heads is crucial for ensuring a pleasant and immersive experience in AR/VR and video conferencing applications.

Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry

1 code implementation CVPR 2023 Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, Zhigang Tu

Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing.

motion retargeting

Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors

no code implementations ICCV 2023 Zhangyang Xiong, Di Kang, Derong Jin, Weikai Chen, Linchao Bao, Shuguang Cui, Xiaoguang Han

Specifically, we bridge the latent space of Get3DHuman with that of StyleGAN-Human via a specially-designed prior network, where the input latent code is mapped to the shape and texture feature volumes spanned by the pixel-aligned 3D reconstructor.

Audio2Gestures: Generating Diverse Gestures from Audio

no code implementations17 Jan 2023 Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Linchao Bao, Zhenyu He

Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline.

Gesture Generation

Learning Audio-Driven Viseme Dynamics for 3D Face Animation

no code implementations15 Jan 2023 Linchao Bao, Haoxian Zhang, Yue Qian, Tangli Xue, Changhai Chen, Xuefei Zhe, Di Kang

We show that the predicted viseme curves can be applied to different viseme-rigged characters to yield various personalized animations with realistic and natural facial motions.

3D Face Animation

CARD: Semantic Segmentation with Efficient Class-Aware Regularized Decoder

1 code implementation11 Jan 2023 Ye Huang, Di Kang, Liang Chen, Wenjing Jia, Xiangjian He, Lixin Duan, Xuefei Zhe, Linchao Bao

Extensive experiments and ablation studies conducted on multiple benchmark datasets demonstrate that the proposed CAR can boost the accuracy of all baseline models by up to 2. 23% mIOU with superior generalization ability.

Representation Learning Semantic Segmentation +1

FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction

1 code implementation CVPR 2023 Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, Linchao Bao

Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image inputs.

3D Face Reconstruction

NEURAL MARIONETTE: A Transformer-based Multi-action Human Motion Synthesis System

no code implementations27 Sep 2022 Weiqiang Wang, Xuefei Zhe, Qiuhong Ke, Di Kang, Tingguang Li, Ruizhi Chen, Linchao Bao

Along with the novel system, we also present a new dataset dedicated to the multi-action motion synthesis task, which contains both action tags and their contextual information.

Motion Synthesis Rolling Shutter Correction +1

Learning to Construct 3D Building Wireframes from 3D Line Clouds

1 code implementation25 Aug 2022 Yicheng Luo, Jing Ren, Xuefei Zhe, Di Kang, Yajing Xu, Peter Wonka, Linchao Bao

The network takes a line cloud as input , i. e., a nonstructural and unordered set of 3D line segments extracted from multi-view images, and outputs a 3D wireframe of the underlying building, which consists of a sparse set of 3D junctions connected by line segments.

Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds

no code implementations14 Jun 2022 Runsong Zhu, Di Kang, Ka-Hei Hui, Yue Qian, Xuefei Zhe, Zhen Dong, Linchao Bao, Pheng-Ann Heng, Chi-Wing Fu

To guide the network quickly fit the coarse shape, we propose to utilize the signed supervision in regions that are obviously outside the object and can be easily determined, resulting in our semi-signed supervision.

Surface Reconstruction

REALY: Rethinking the Evaluation of 3D Face Reconstruction

1 code implementation18 Mar 2022 Zenghao Chai, Haoxian Zhang, Jing Ren, Di Kang, Zhengzhuo Xu, Xuefei Zhe, Chun Yuan, Linchao Bao

The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan.

3D Face Reconstruction

CAR: Class-aware Regularizations for Semantic Segmentation

1 code implementation arXiv:2203.07160 2022 Ye Huang, Di Kang, Liang Chen, Xuefei Zhe, Wenjing Jia, Xiangjian He, Linchao Bao

Recent segmentation methods, such as OCR and CPNet, utilizing "class level" information in addition to pixel features, have achieved notable success for boosting the accuracy of existing network modules.

Representation Learning Semantic Segmentation

NeRFReN: Neural Radiance Fields with Reflections

no code implementations CVPR 2022 Yuan-Chen Guo, Di Kang, Linchao Bao, Yu He, Song-Hai Zhang

Specifically, we propose to split a scene into transmitted and reflected components, and model the two components with separate neural radiance fields.

Depth Estimation Novel View Synthesis

ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

1 code implementation26 Sep 2021 Yahui Liu, Yajing Chen, Linchao Bao, Nicu Sebe, Bruno Lepri, Marco De Nadai

The ISF manipulates the semantics of an input latent code to make the image generated from it lying in the desired visual domain.

Image-to-Image Translation Translation

Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders

no code implementations ICCV 2021 Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Zhenyu He, Linchao Bao

In order to overcome this problem, we propose a novel conditional variational autoencoder (VAE) that explicitly models one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code.

Gesture Generation

UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing

no code implementations12 Aug 2021 Meng Cao, HaoZhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo

Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.

3D Reconstruction Face Reenactment +3

Animatable Neural Radiance Fields from Monocular RGB Videos

1 code implementation25 Jun 2021 Jianchuan Chen, Ying Zhang, Di Kang, Xuefei Zhe, Linchao Bao, Xu Jia, Huchuan Lu

We present animatable neural radiance fields (animatable NeRF) for detailed human avatar creation from monocular videos.

3D Human Reconstruction Neural Rendering +2

Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation

no code implementations CVPR 2021 Yahui Liu, Enver Sangineto, Yajing Chen, Linchao Bao, Haoxian Zhang, Nicu Sebe, Bruno Lepri, Wei Wang, Marco De Nadai

In this paper, we propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which: 1) Both intra- and inter-domain interpolations correspond to gradual changes in the generated images and 2) The content of the source image is better preserved during the translation.

Translation Unsupervised Image-To-Image Translation

Model-based 3D Hand Reconstruction via Self-Supervised Learning

1 code implementation CVPR 2021 Yujin Chen, Zhigang Tu, Di Kang, Linchao Bao, Ying Zhang, Xuefei Zhe, Ruizhi Chen, Junsong Yuan

For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations.

Self-Supervised Learning

High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies

2 code implementations12 Oct 2020 Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, HaoZhi Huang, Xinwei Jiang, Jue Wang, Dong Yu, Zhengyou Zhang

We present a fully automatic system that can produce high-fidelity, photo-realistic 3D digital human heads with a consumer RGB-D selfie camera.

Vocal Bursts Intensity Prediction

Self-supervised Video Representation Learning by Uncovering Spatio-temporal Statistics

2 code implementations31 Aug 2020 Jiangliu Wang, Jianbo Jiao, Linchao Bao, Shengfeng He, Wei Liu, Yun-hui Liu

Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc.

Action Recognition Representation Learning +3

Task-agnostic Temporally Consistent Facial Video Editing

no code implementations3 Jul 2020 Meng Cao, Hao-Zhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo

Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.

3D Reconstruction Video Editing

Laplacian Denoising Autoencoder

no code implementations30 Mar 2020 Jianbo Jiao, Linchao Bao, Yunchao Wei, Shengfeng He, Honghui Shi, Rynson Lau, Thomas S. Huang

This can be naturally generalized to span multiple scales with a Laplacian pyramid representation of the input data.

Denoising Self-Supervised Learning

Self-supervised Learning of Detailed 3D Face Reconstruction

1 code implementation25 Oct 2019 Yajing Chen, Fanzi Wu, Zeyu Wang, Yibing Song, Yonggen Ling, Linchao Bao

The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage.

3D Face Reconstruction Face Alignment +1

MVF-Net: Multi-View 3D Face Morphable Model Regression

1 code implementation CVPR 2019 Fanzi Wu, Linchao Bao, Yajing Chen, Yonggen Ling, Yibing Song, Songnan Li, King Ngi Ngan, Wei Liu

The main ingredient of the view alignment loss is a differentiable dense optical flow estimator that can backpropagate the alignment errors between an input view and a synthetic rendering from another input view, which is projected to the target view through the 3D shape to be inferred.

Optical Flow Estimation regression

Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

no code implementations22 Nov 2018 Yibing Song, Jiawei Zhang, Lijun Gong, Shengfeng He, Linchao Bao, Jinshan Pan, Qingxiong Yang, Ming-Hsuan Yang

We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image.

Deblurring Face Hallucination +2

Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks

1 code implementation CVPR 2018 Jiawei Zhang, Jinshan Pan, Jimmy Ren, Yibing Song, Linchao Bao, Rynson W. H. Lau, Ming-Hsuan Yang

The proposed network is composed of three deep convolutional neural networks (CNNs) and a recurrent neural network (RNN).

Ranked #10 on Deblurring on RealBlur-R (trained on GoPro) (SSIM (sRGB) metric)


VITAL: VIsual Tracking via Adversarial Learning

no code implementations CVPR 2018 Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, WangMeng Zuo, Chunhua Shen, Rynson Lau, Ming-Hsuan Yang

To augment positive samples, we use a generative network to randomly generate masks, which are applied to adaptively dropout input features to capture a variety of appearance changes.

General Classification Visual Tracking

CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRF

no code implementations CVPR 2018 Linchao Bao, Baoyuan Wu, Wei Liu

With temporal dependencies established by optical flow, the resulting MRF model combines both spatial and temporal cues for tackling video object segmentation.

Object One-Shot Segmentation +4

Stylizing Face Images via Multiple Exemplars

no code implementations28 Aug 2017 Yibing Song, Linchao Bao, Shengfeng He, Qingxiong Yang, Ming-Hsuan Yang

We address the problem of transferring the style of a headshot photo to face images.

Fast Preprocessing for Robust Face Sketch Synthesis

no code implementations1 Aug 2017 Yibing Song, Jiawei Zhang, Linchao Bao, Qingxiong Yang

Exemplar-based face sketch synthesis methods usually meet the challenging problem that input photos are captured in different lighting conditions from training photos.

Face Sketch Synthesis

Robust Piecewise-Constant Smoothing: M-Smoother Revisited

no code implementations28 Oct 2014 Linchao Bao, Qingxiong Yang

In addition, high-quality piecewise-constant smoothing can be achieved via a number of bilateral filtering or guided filtering integrated in the proposed framework.


Cannot find the paper you are looking for? You can Submit a new open access paper.