Search Results for author: Jiaolong Yang

Found 42 papers, 21 papers with code

VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time

no code implementations16 Apr 2024 Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, Baining Guo

We introduce VASA, a framework for generating lifelike talking faces with appealing visual affective skills (VAS) given a single static image and a speech audio clip.

GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling

no code implementations28 Mar 2024 BoWen Zhang, Yiji Cheng, Jiaolong Yang, Chunyu Wang, Feng Zhao, Yansong Tang, Dong Chen, Baining Guo

We introduce a radiance representation that is both structured and fully explicit and thus greatly facilitates 3D generative modeling.

Decoder Text to 3D

Diffusion Models are Geometry Critics: Single Image 3D Editing Using Pre-Trained Diffusion Priors

no code implementations18 Mar 2024 Ruicheng Wang, Jianfeng Xiang, Jiaolong Yang, Xin Tong

We propose a novel image editing technique that enables 3D manipulations on single images, such as object rotation and translation.

Novel View Synthesis

A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars

1 code implementation9 Jan 2024 Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, Xin Tong

The objective of this paper is to develop a functional system for translating spoken languages into sign languages, referred to as Spoken2Sign translation.

Sign Language Translation Translation

Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs

1 code implementation12 Dec 2023 Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, Chong Luo

This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision.

Novel View Synthesis Pose Estimation

AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections

no code implementations5 Sep 2023 Yue Wu, Sicheng Xu, Jianfeng Xiang, Fangyun Wei, Qifeng Chen, Jiaolong Yang, Xin Tong

For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations.

3D-aware Image Generation using 2D Diffusion Models

no code implementations ICCV 2023 Jianfeng Xiang, Jiaolong Yang, Binbin Huang, Xin Tong

In this paper, we introduce a novel 3D-aware image generation method that leverages 2D diffusion models.

Image Generation

RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch

no code implementations28 Feb 2023 Yizhong Zhang, Zhiqi Li, Sicheng Xu, Chong Li, Jiaolong Yang, Xin Tong, Baining Guo

A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen.

NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation

no code implementations CVPR 2023 Yu Yin, Kamran Ghasedi, HsiangTao Wu, Jiaolong Yang, Xin Tong, Yun Fu

Furthermore, our method leverages explicit and implicit 3D regularizations using the in-domain neighborhood samples around the optimized latent code to remove geometrical and visual artifacts.

Image Animation

A Structure-Guided Diffusion Model for Large-Hole Image Completion

1 code implementation18 Nov 2022 Daichi Horita, Jiaolong Yang, Dong Chen, Yuki Koyama, Kiyoharu Aizawa, Nicu Sebe

The structure generator generates an edge image representing plausible structures within the holes, which is then used for guiding the texture generation process.

Denoising Texture Synthesis

AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars

1 code implementation12 Oct 2022 Yue Wu, Yu Deng, Jiaolong Yang, Fangyun Wei, Qifeng Chen, Xin Tong

To achieve meaningful control over facial expressions via deformation, we propose a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the 3D-aware GAN.

Disentanglement Face Model +1

Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects

no code implementations9 Sep 2022 Ziyu Wang, Yu Deng, Jiaolong Yang, Jingyi Yu, Xin Tong

Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e. g., chairs) with large topological variance.

Disentanglement Image Generation +1

Deep Deformable 3D Caricatures with Learned Shape Control

1 code implementation29 Jul 2022 Yucheol Jung, Wonjong Jang, Soongjin Kim, Jiaolong Yang, Xin Tong, Seungyong Lee

To achieve the goal, we propose an MLP-based framework for building a deformable surface model, which takes a latent code and produces a 3D surface.

Caricature Position

GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifolds

no code implementations ICCV 2023 Jianfeng Xiang, Jiaolong Yang, Yu Deng, Xin Tong

This paper proposes a novel 3D-aware GAN that can generate high resolution images (up to 1024X1024) while keeping strict 3D consistency as in volume rendering.

Image Generation Super-Resolution

Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images

1 code implementation24 May 2022 Yuxuan Han, Ruicheng Wang, Jiaolong Yang

This paper deals with the challenging task of synthesizing novel views for in-the-wild photographs.

Monocular Depth Estimation

Real-Time Neural Character Rendering with Pose-Guided Multiplane Images

1 code implementation25 Apr 2022 Hao Ouyang, Bo Zhang, Pan Zhang, Hao Yang, Jiaolong Yang, Dong Chen, Qifeng Chen, Fang Wen

We propose pose-guided multiplane image (MPI) synthesis which can render an animatable character in real scenes with photorealistic quality.

Image-to-Image Translation Neural Rendering +1

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

no code implementations31 Mar 2022 Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, Xin Tong

For this task, we propose a simple yet effective method to train a generalizable NeRF with multiview images as conditional input.

Novel View Synthesis

VirtualCube: An Immersive 3D Video Communication System

no code implementations13 Dec 2021 Yizhong Zhang, Jiaolong Yang, Zhen Liu, Ruicheng Wang, Guojun Chen, Xin Tong, Baining Guo

The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies.

Depth Estimation

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation

1 code implementation9 Jul 2021 Wonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee

Our framework, dubbed StyleCariGAN, automatically creates a realistic and detailed caricature from an input photo with optional controls on shape exaggeration degree and color stylization type.

Caricature

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search

1 code implementation26 May 2021 Yuxuan Han, Jiaolong Yang, Ying Fu

We further propose a Disentanglement-Transformation (DT) metric to quantify the attribute transformation and disentanglement efficacy and find the optimal control factor between attribute-level and instance-specific directions based on it.

Attribute Disentanglement

High-Resolution Optical Flow from 1D Attention and Correlation

1 code implementation ICCV 2021 Haofei Xu, Jiaolong Yang, Jianfei Cai, Juyong Zhang, Xin Tong

Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images.

4k Optical Flow Estimation +1

Learning High-Fidelity Face Texture Completion Without Complete Face Texture

no code implementations ICCV 2021 Jongyoo Kim, Jiaolong Yang, Xin Tong

For face texture completion, previous methods typically use some complete textures captured by multiview imaging systems or 3D scanners for supervised learning.

Vocal Bursts Intensity Prediction

Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence

1 code implementation CVPR 2021 Yu Deng, Jiaolong Yang, Xin Tong

We propose a novel Deformed Implicit Field (DIF) representation for modeling 3D shapes of a category and generating dense correspondences among shapes.

Deep 3D Portrait from a Single Image

1 code implementation CVPR 2020 Sicheng Xu, Jiaolong Yang, Dong Chen, Fang Wen, Yu Deng, Yunde Jia, Xin Tong

We evaluate the accuracy of our method both in 3D and with pose manipulation tasks on 2D images.

Face Model Stereo Matching

A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising

1 code implementation CVPR 2020 Kaixuan Wei, Ying Fu, Jiaolong Yang, Hua Huang

Lacking rich and realistic data, learned single image denoising algorithms generalize poorly to real raw images that do not resemble the data used for training.

Image Denoising

Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements

1 code implementation CVPR 2019 Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, Hua Huang

Removing undesirable reflections from a single image captured through a glass window is of practical importance to visual computing systems.

Reflection Removal

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set

4 code implementations20 Mar 2019 Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, Xin Tong

Recently, deep learning based 3D face reconstruction methods have shown promising results in both quality and efficiency. However, training deep neural networks typically requires a large volume of data, whereas face images with ground-truth 3D face shapes are scarce.

Ranked #3 on 3D Face Reconstruction on Florence (RMSE Cooperative metric)

3D Face Reconstruction Weakly-supervised Learning

Image Smoothing via Unsupervised Learning

1 code implementation7 Nov 2018 Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, Xin Tong

Image smoothing represents a fundamental component of many disparate computer vision and graphics applications.

Image Manipulation image smoothing

Deep Single-View 3D Object Reconstruction with Visual Hull Embedding

1 code implementation10 Sep 2018 Hanqing Wang, Jiaolong Yang, Wei Liang, Xin Tong

The key idea of our method is to leverage object mask and pose estimation from CNNs to assist the 3D shape learning by constructing a probabilistic single-view visual hull inside of the network.

3D Object Reconstruction Object +1

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

1 code implementation ECCV 2018 Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model.

Quantization

Fast, Accurate Thin-Structure Obstacle Detection for Autonomous Mobile Robots

no code implementations14 Aug 2017 Chen Zhou, Jiaolong Yang, Chunshui Zhao, Gang Hua

This work is devoted to a task that is indispensable for safety yet was largely overlooked in the past -- detecting obstacles that are of very thin structures, such as wires, cables and tree branches.

Self-Driving Cars Visual Odometry

A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing

1 code implementation ICCV 2017 Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf

This paper proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering.

image smoothing Reflection Removal +1

Revisiting Deep Intrinsic Image Decompositions

no code implementations CVPR 2018 Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf

While invaluable for many computer vision applications, decomposing a natural image into intrinsic reflectance and shading layers represents a challenging, underdetermined inverse problem.

Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration

no code implementations11 May 2016 Jiaolong Yang, Hongdong Li, Dylan Campbell, Yunde Jia

The evaluation demonstrates that the proposed method is able to produce reliable registration results regardless of the initialization.

Image to Point Cloud Registration

Robust Optical Flow Estimation of Double-Layer Images under Transparency or Reflection

no code implementations CVPR 2016 Jiaolong Yang, Hongdong Li, Yuchao Dai, Robby T. Tan

This paper deals with a challenging, frequently encountered, yet not properly investigated problem in two-frame optical flow estimation.

Optical Flow Estimation valid

Neural Aggregation Network for Video Face Recognition

no code implementations CVPR 2017 Jiaolong Yang, Peiran Ren, Dong-Qing Zhang, Dong Chen, Fang Wen, Hongdong Li, Gang Hua

The network takes a face video or face image set of a person with a variable number of face images as its input, and produces a compact, fixed-dimension feature representation for recognition.

Face Identification Face Recognition +1

Cannot find the paper you are looking for? You can Submit a new open access paper.