Search Results for author: Juyong Zhang

Found 59 papers, 28 papers with code

DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction

no code implementations18 Mar 2024 Yuxin Yao, Siyu Ren, Junhui Hou, Zhi Deng, Juyong Zhang, Wenping Wang

Furthermore, we propose a learnable deformation representation based on the learnable control points and blending weights, which can deform the template surface non-rigidly while maintaining the consistency of the local shape.

Surface Reconstruction

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web

no code implementations27 Dec 2023 Kaiwen Song, Juyong Zhang

Meanwhile, we carefully design the training and inference process such that the final rendering result on web is consistent with training.

3D Scene Reconstruction Management +1

HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video

no code implementations14 Dec 2023 Xueying Wang, Juyong Zhang

We further model the head geometry in the canonical space with a learnable signed distance field (SDF) and optimize it using the volumetric rendering with the guidance of two-main head priors to improve the reconstruction accuracy and robustness.

FlashAvatar: High-Fidelity Digital Avatar Rendering at 300FPS

no code implementations3 Dec 2023 Jun Xiang, Xuan Gao, Yudong Guo, Juyong Zhang

We propose FlashAvatar, a novel and lightweight 3D animatable avatar representation that could reconstruct a digital avatar from a short monocular video sequence in minutes and render high-fidelity photo-realistic images at 300FPS on a consumer-grade GPU.

Face Model

CosAvatar: Consistent and Animatable Portrait Video Tuning with Text Prompt

no code implementations30 Nov 2023 Haiyao Xiao, Chenglai Zhong, Xuan Gao, Yudong Guo, Juyong Zhang

Recently, text-guided digital portrait editing has attracted more and more attentions.

$L_0$-Sampler: An $L_{0}$ Model Guided Volume Sampling for NeRF

no code implementations13 Nov 2023 Liangchen Li, Juyong Zhang

Since being proposed, Neural Radiance Fields (NeRF) have achieved great success in related tasks, mainly adopting the hierarchical volume sampling (HVS) strategy for volume rendering.

3D Reconstruction

MetaHead: An Engine to Create Realistic Digital Head

no code implementations3 Apr 2023 Dingyun Zhang, Chenglai Zhong, Yudong Guo, Yang Hong, Juyong Zhang

Experiments validate that our controllable digital head engine achieves the state-of-the-art generation visual quality and reconstruction accuracy.

Image Generation

IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF

no code implementations28 Feb 2023 Bo Peng, Jun Hu, Jingtao Zhou, Xuan Gao, Juyong Zhang

To achieve this target, we introduce a continuous and optimizable intrinsic coordinate rather than the original explicit Euclidean coordinate in the hash encoding module of instant-NGP.

Novel View Synthesis

Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud Analysis

1 code implementation17 Dec 2022 Qijian Zhang, Junhui Hou, Yue Qian, Yiming Zeng, Juyong Zhang, Ying He

In this paper, we present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology as a completely regular 2D point geometry image (PGI) structure, in which coordinates of spatial points are captured in colors of image pixels.

Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video

1 code implementation12 Oct 2022 Xuan Gao, Chenglai Zhong, Jun Xiang, Yang Hong, Yudong Guo, Juyong Zhang

We present a novel semantic model for human head defined with neural radiance field.

SelfNeRF: Fast Training NeRF for Human from Monocular Self-rotating Video

no code implementations4 Oct 2022 Bo Peng, Jun Hu, Jingtao Zhou, Juyong Zhang

Extensive experimental results on several different datasets demonstrate the effectiveness and efficiency of SelfNeRF to challenging monocular videos.

Novel View Synthesis

Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera

1 code implementation30 Jun 2022 Hongrui Cai, Wanquan Feng, Xuetao Feng, Yan Wang, Juyong Zhang

We propose Neural-DynamicReconstruction (NDR), a template-free method to recover high-fidelity geometry and motions of a dynamic scene from a monocular RGB-D camera.

Dynamic Reconstruction Monocular Reconstruction +2

Fast and Robust Non-Rigid Registration Using Accelerated Majorization-Minimization

1 code implementation7 Jun 2022 Yuxin Yao, Bailin Deng, Weiwei Xu, Juyong Zhang

In this paper, we propose a formulation for robust non-rigid registration based on a globally smooth robust norm for alignment and regularization, which can effectively handle outliers and partial overlaps.

A Survey of Non-Rigid 3D Registration

no code implementations11 Mar 2022 Bailin Deng, Yuxin Yao, Roberto M. Dyke, Juyong Zhang

Non-rigid registration computes an alignment between a source surface with a target surface in a non-rigid manner.

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

1 code implementation CVPR 2022 Boyi Jiang, Yang Hong, Hujun Bao, Juyong Zhang

Meanwhile, the explicit mesh is updated periodically to adjust its topology changes, and a consistency loss is designed to match both representations.

Neural Rendering

Sketch2PQ: Freeform Planar Quadrilateral Mesh Design via a Single Sketch

no code implementations23 Jan 2022 Zhi Deng, Yang Liu, Hao Pan, Wassim Jabi, Juyong Zhang, Bailin Deng

In this work, we present a novel sketch-based system to bridge the concept design and digital modeling of freeform roof-like shapes represented as planar quadrilateral (PQ) meshes.

HeadNeRF: A Real-time NeRF-based Parametric Head Model

1 code implementation CVPR 2022 Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, Juyong Zhang

Different from existing related parametric models, we use the neural radiance fields as a novel 3D proxy instead of the traditional 3D textured mesh, which makes that HeadNeRF is able to generate high fidelity images.

Neural Rendering

Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling

1 code implementation CVPR 2022 Wanquan Feng, Jin Li, Hongrui Cai, Xiaonan Luo, Juyong Zhang

Different from traditional point cloud representation where each point only represents a position or a local plane in the 3D space, each point in Neural Points represents a local continuous geometric shape via neural fields.

A Robust Loss for Point Cloud Registration

1 code implementation ICCV 2021 Zhi Deng, Yuxin Yao, Bailin Deng, Juyong Zhang

The performance of surface registration relies heavily on the metric used for the alignment error between the source and target shapes.

Point Cloud Registration

Prior-Guided Multi-View 3D Head Reconstruction

no code implementations9 Jul 2021 Xueying Wang, Yudong Guo, Zhongqi Yang, Juyong Zhang

Extensive ablation studies and comparisons with state-of-the-art methods demonstrate that our method can generate high-fidelity 3D head geometries with the guidance of these priors.

Neural Rendering Semantic Segmentation

High-Resolution Optical Flow from 1D Attention and Correlation

1 code implementation ICCV 2021 Haofei Xu, Jiaolong Yang, Jianfei Cai, Juyong Zhang, Xin Tong

Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images.

Optical Flow Estimation Vocal Bursts Intensity Prediction

StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision

1 code implementation CVPR 2021 Yang Hong, Juyong Zhang, Boyi Jiang, Yudong Guo, Ligang Liu, Hujun Bao

In this paper, we propose StereoPIFu, which integrates the geometric constraints of stereo vision with implicit function representation of PIFu, to recover the 3D shape of the clothed human from a pair of low-cost rectified images.

AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis

1 code implementation ICCV 2021 Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, Juyong Zhang

Generating high-fidelity talking head video by fitting with the input audio sequence is a challenging problem that receives considerable attentions recently.

Talking Face Generation

ParaNet: Deep Regular Representation for 3D Point Clouds

no code implementations5 Dec 2020 Qijian Zhang, Junhui Hou, Yue Qian, Juyong Zhang, Ying He

Although convolutional neural networks have achieved remarkable success in analyzing 2D images/videos, it is still non-trivial to apply the well-developed 2D techniques in regular domains to the irregular 3D point cloud data.

Recurrent Multi-view Alignment Network for Unsupervised Surface Registration

1 code implementation CVPR 2021 Wanquan Feng, Juyong Zhang, Hongrui Cai, Haofei Xu, Junhui Hou, Hujun Bao

Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data.

Deformable Object Manipulation Neural Rendering +1

Modeling Caricature Expressions by 3D Blendshape and Dynamic Texture

no code implementations13 Aug 2020 Keyu Chen, Jianmin Zheng, Jianfei Cai, Juyong Zhang

The problem of deforming an artist-drawn caricature according to a given normal face expression is of interest in applications such as social media, animation and entertainment.

Caricature Generative Adversarial Network +1

Facial Expression Retargeting from Human to Avatar Made Easy

no code implementations12 Aug 2020 Juyong Zhang, Keyu Chen, Jianmin Zheng

Then we construct correspondences between the two latent spaces guided by geometric and perceptual constraints.

Geometric Matching Translation

Fast and Robust Iterative Closest Point

1 code implementation15 Jul 2020 Juyong Zhang, Yuxin Yao, Bailin Deng

On challenging datasets with noises and partial overlaps, we achieve similar or better accuracy than Sparse ICP while being at least an order of magnitude faster.

3D Reconstruction

Contextualized Graph Attention Network for Recommendation with Item Knowledge Graph

no code implementations24 Apr 2020 Susen Yang, Yong liu, Yonghui Xu, Chunyan Miao, Min Wu, Juyong Zhang

Graph neural networks (GNN) have recently been applied to exploit knowledge graph (KG) for recommendation.

Graph Attention

Learning Hierarchical Review Graph Representations for Recommendation

no code implementations24 Apr 2020 Yong liu, Susen Yang, Yinan Zhang, Chunyan Miao, Zaiqing Nie, Juyong Zhang

Therefore, they may not be effective in capturing the global dependency between words, and tend to be easily biased by noise review information.

Graph Attention

Landmark Detection and 3D Face Reconstruction for Caricature using a Nonlinear Parametric Model

1 code implementation20 Apr 2020 Hongrui Cai, Yudong Guo, Zhuang Peng, Juyong Zhang

To this end, we first build a dataset with various styles of 2D caricatures and their corresponding 3D shapes, and then build a parametric model on vertex based deformation space for 3D caricature face.

3D Face Reconstruction Caricature

Quasi-Newton Solver for Robust Non-Rigid Registration

1 code implementation CVPR 2020 Yuxin Yao, Bailin Deng, Weiwei Xu, Juyong Zhang

Imperfect data (noise, outliers and partial overlap) and high degrees of freedom make non-rigid registration a classical challenging problem in computer vision.

BCNet: Learning Body and Cloth Shape from A Single Image

1 code implementation ECCV 2020 Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, Hujun Bao

In this paper, we consider the problem to automatically reconstruct garment and body shapes from a single near-front view RGB image.

Lightweight Photometric Stereo for Facial Details Recovery

1 code implementation CVPR 2020 Xueying Wang, Yudong Guo, Bailin Deng, Juyong Zhang

Recently, 3D face reconstruction from a single image has achieved great success with the help of deep learning and shape prior knowledge, but they often fail to produce accurate geometry details.

3D Face Reconstruction Data Augmentation

3D-CariGAN: An End-to-End Solution to 3D Caricature Generation from Face Photos

1 code implementation15 Mar 2020 Zipeng Ye, Mengfei Xia, Yanan sun, Ran Yi, MinJing Yu, Juyong Zhang, Yu-Kun Lai, Yong-Jin Liu

The most challenging issue for our system is that the source domain of face photos (characterized by normal 2D faces) is significantly different from the target domain of 3D caricatures (characterized by 3D exaggerated face shapes and textures).

Caricature

Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose

1 code implementation24 Feb 2020 Ran Yi, Zipeng Ye, Juyong Zhang, Hujun Bao, Yong-Jin Liu

In this paper, we address this problem by proposing a deep neural network model that takes an audio signal A of a source person and a very short video V of a target person as input, and outputs a synthesized high-quality talking face video with personalized head pose (making use of the visual information in V), expression and lip synchronization (by considering both A and V).

3D Face Animation Video Generation

Semi-supervised 3D Face Reconstruction with Nonlinear Disentangled Representations

no code implementations25 Sep 2019 Zhongpai Gao, Juyong Zhang, Yudong Guo, Chao Ma, Guangtao Zhai, Xiaokang Yang

Moreover, the identity and expression representations are entangled in these models, which hurdles many facial editing applications.

3D Face Reconstruction Facial Editing

Bandit Learning for Diversified Interactive Recommendation

no code implementations1 Jul 2019 Yong Liu, Yingtai Xiao, Qiong Wu, Chunyan Miao, Juyong Zhang

Interactive recommender systems that enable the interactions between users and the recommender system have attracted increasing research attentions.

Bayesian Inference Recommendation Systems +1

3D Magic Mirror: Automatic Video to 3D Caricature Translation

no code implementations3 Jun 2019 Yudong Guo, Luo Jiang, Lin Cai, Juyong Zhang

Caricature is an abstraction of a real person which distorts or exaggerates certain features, but still retains a likeness.

Caricature Translation

Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network

1 code implementation14 May 2019 Boyi Jiang, Juyong Zhang, Jianfei Cai, Jianmin Zheng

Human bodies exhibit various shapes for different identities or poses, but the body shape has certain similarities in structure and thus can be embedded in a low-dimensional space.

Representation Learning

Region Deformer Networks for Unsupervised Depth Estimation from Unconstrained Monocular Videos

1 code implementation26 Feb 2019 Haofei Xu, Jianmin Zheng, Jianfei Cai, Juyong Zhang

In this paper, we propose a new learning based method consisting of DepthNet, PoseNet and Region Deformer Networks (RDN) to estimate depth from unconstrained monocular videos without ground truth supervision.

Depth Estimation

Real-time 3D Face-Eye Performance Capture of a Person Wearing VR Headset

no code implementations21 Jan 2019 Guoxian Song, Jianfei Cai, Tat-Jen Cham, Jianmin Zheng, Juyong Zhang, Henry Fuchs

Teleconference or telepresence based on virtual reality (VR) headmount display (HMD) device is a very interesting and promising application since HMD can provide immersive feelings for users.

Parallel and Scalable Heat Methods for Geodesic Distance Computation

1 code implementation14 Dec 2018 Jiong Tao, Juyong Zhang, Bailin Deng, Zheng Fang, Yue Peng, Ying He

In this paper, we propose a parallel and scalable approach for geodesic distance computation on triangle meshes.

Graphics

Robust RGB-D Face Recognition Using Attribute-Aware Loss

no code implementations24 Nov 2018 Luo Jiang, Juyong Zhang, Bailin Deng

Existing convolutional neural network (CNN) based face recognition algorithms typically learn a discriminative feature mapping, using a loss function that enforces separation of features from different classes and/or aggregation of features within the same class.

Attribute Face Recognition

3D Face From X: Learning Face Shape from Diverse Sources

1 code implementation16 Aug 2018 Yudong Guo, Lin Cai, Juyong Zhang

Although 3D scanned data contain accurate geometric information of face shapes, the capture system is expensive and such datasets usually contain a small number of subjects.

3D Face Modelling 3D Face Reconstruction +1

Fast K-Means Clustering with Anderson Acceleration

no code implementations27 May 2018 Juyong Zhang, Yuxin Yao, Yue Peng, Hao Yu, Bailin Deng

We propose a novel method to accelerate Lloyd's algorithm for K-Means clustering.

Clustering

Alive Caricature from 2D to 3D

1 code implementation CVPR 2018 Qianyi Wu, Juyong Zhang, Yu-Kun Lai, Jianmin Zheng, Jianfei Cai

Caricature is an art form that expresses subjects in abstract, simple and exaggerated view.

Caricature

Conditional Adversarial Synthesis of 3D Facial Action Units

no code implementations21 Feb 2018 Zhilei Liu, Guoxian Song, Jianfei Cai, Tat-Jen Cham, Juyong Zhang

Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training.

Data Augmentation Image Generation

Static/Dynamic Filtering for Mesh Geometry

2 code implementations10 Dec 2017 Juyong Zhang, Bailin Deng, Yang Hong, Yue Peng, Wenjie Qin, Ligang Liu

Extensive experimental results demonstrate the effectiveness of the proposed filter for various geometry processing applications such as mesh denoising, geometry feature enhancement, and texture color filtering.

Graphics

Deep Face Feature for Face Alignment

no code implementations9 Aug 2017 Boyi Jiang, Juyong Zhang, Bailin Deng, Yudong Guo, Ligang Liu

The deep face feature (DFF) is trained using correspondence between face images rendered from different views.

Face Alignment Inverse Rendering +1

CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images

no code implementations3 Aug 2017 Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang, Jianmin Zheng

With the powerfulness of convolution neural networks (CNN), CNN based face reconstruction has recently shown promising performance in reconstructing detailed face shape from 2D face images.

3D Face Reconstruction Face Model +1

3D Face Reconstruction with Geometry Details from a Single Image

no code implementations18 Feb 2017 Luo Jiang, Juyong Zhang, Bailin Deng, Hao Li, Ligang Liu

3D face reconstruction from a single image is a classical and challenging problem, with wide applications in many areas.

3D Face Reconstruction Face Model

Recovering Surface Details under General Unknown Illumination Using Shading and Coarse Multi-view Stereo

no code implementations CVPR 2014 Di Xu, Qi Duan, Jianming Zheng, Juyong Zhang, Jianfei Cai, Tat-Jen Cham

As a result, our approach is robust, stable and is able to efficiently recover high quality of surface details even starting with a coarse MVS.

Cannot find the paper you are looking for? You can Submit a new open access paper.