Search Results for author: Wenping Wang

Found 115 papers, 46 papers with code

TANet: Towards Fully Automatic Tooth Arrangement

1 code implementation ECCV 2020 Guodong Wei, Zhiming Cui, Yumeng Liu, Nenglun Chen, Runnan Chen, Guiqing Li, Wenping Wang

Determining optimal target tooth arrangements is a key step of treatment planning in digital orthodontics.

Pose Prediction

Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation

1 code implementation ACL 2022 Guanhua Chen, Shuming Ma, Yun Chen, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

When applied to zero-shot cross-lingual abstractive summarization, it produces an average performance gain of 12. 3 ROUGE-L over mBART-ft. We conduct detailed analyses to understand the key ingredients of SixT+, including multilinguality of the auxiliary parallel data, positional disentangled encoder, and the cross-lingual transferability of its encoder.

Abstractive Text Summarization Cross-Lingual Abstractive Summarization +5

Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing

no code implementations24 Mar 2024 Yongqing Liang, Congyi Zhang, Junli Zhao, Wenping Wang, Xin Li

Existing methods for automated facial reconstruction yield inaccurate results, suffering from the non-determinative nature of the problem that a skull with a sparse set of tissue depth cannot fully determine the skinned face.

3D Face Reconstruction Anatomy

Segment Anything Model for Road Network Graph Extraction

1 code implementation24 Mar 2024 Congrui Hetang, Haoru Xue, Cindy Le, Tianwei Yue, Wenping Wang, Yihui He

We propose SAM-Road, an adaptation of the Segment Anything Model (SAM) for extracting large-scale, vectorized road network graphs from satellite imagery.

Graph Learning Semantic Segmentation

DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction

no code implementations18 Mar 2024 Yuxin Yao, Siyu Ren, Junhui Hou, Zhi Deng, Juyong Zhang, Wenping Wang

Furthermore, we propose a learnable deformation representation based on the learnable control points and blending weights, which can deform the template surface non-rigidly while maintaining the consistency of the local shape.

Surface Reconstruction

Semantic Human Mesh Reconstruction with Textures

no code implementations5 Mar 2024 Xiaoyu Zhan, Jianxin Yang, Yuanqi Li, Jie Guo, Yanwen Guo, Wenping Wang

SHERT applies semantic- and normal-based sampling between the detailed surface (e. g. mesh and SDF) and the corresponding SMPL-X model to obtain a partially sampled semantic mesh and then generates the complete semantic mesh by our specifically designed self-supervised completion and refinement networks.

Dynamic 3D Point Cloud Sequences as 2D Videos

no code implementations2 Mar 2024 Yiming Zeng, Junhui Hou, Qijian Zhang, Siyu Ren, Wenping Wang

The structured nature of our SPCV representation allows for the seamless adaptation of well-established 2D image/video techniques, enabling efficient and effective processing and analysis of 3D point cloud sequences.

Action Recognition Self-Supervised Learning

GaussianPro: 3D Gaussian Splatting with Progressive Propagation

no code implementations22 Feb 2024 Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, Xuejin Chen

The advent of 3D Gaussian Splatting (3DGS) has recently brought about a revolution in the field of neural rendering, facilitating high-quality renderings at real-time speed.

Neural Rendering Patch Matching

RealDex: Towards Human-like Grasping for Robotic Dexterous Hand

no code implementations21 Feb 2024 Yumeng Liu, Yaxun Yang, Youzhuo Wang, Xiaofei Wu, Jiamin Wang, Yichen Yao, Sören Schwertfeger, Sibei Yang, Wenping Wang, Jingyi Yu, Xuming He, Yuexin Ma

In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data.

Neuromorphic Synergy for Video Binarization

1 code implementation20 Feb 2024 ShiJie Lin, Xiang Zhang, Lei Yang, Lei Yu, Bin Zhou, Xiaowei Luo, Wenping Wang, Jia Pan

We also develop an efficient integration method to propagate this binary image to high frame rate binary video.

Binarization Camera Calibration +1

Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images

no code implementations8 Feb 2024 Xiaoxiao Long, Yuhang Zheng, Yupeng Zheng, Beiwen Tian, Cheng Lin, Lingjie Liu, Hao Zhao, Guyue Zhou, Wenping Wang

We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context.

Depth Estimation

Measuring the Discrepancy between 3D Geometric Models using Directional Distance Fields

no code implementations18 Jan 2024 Siyu Ren, Junhui Hou, Xiaodong Chen, Hongkai Xiong, Wenping Wang

We then transfer the discrepancy between two 3D geometric models as the discrepancy between their DDFs defined on an identical domain, naturally establishing model correspondence.

Scene Flow Estimation

On Optimal Sampling for Learning SDF Using MLPs Equipped with Positional Encoding

no code implementations2 Jan 2024 Guying Lin, Lei Yang, YuAn Liu, Congyi Zhang, Junhui Hou, Xiaogang Jin, Taku Komura, John Keyser, Wenping Wang

Sampling against this intrinsic frequency following the Nyquist-Sannon sampling theorem allows us to determine an appropriate training sampling rate.

Disentangled Clothed Avatar Generation from Text Descriptions

no code implementations8 Dec 2023 Jionghao Wang, YuAn Liu, Zhiyang Dou, Zhengming Yu, Yongqing Liang, Xin Li, Wenping Wang, Rong Xie, Li Song

In this paper, we introduced a novel text-to-avatar generation method that separately generates the human body and the clothes and allows high-quality animation on the generated avatar.

Virtual Try-on

DiffusionPhase: Motion Diffusion in Frequency Domain

no code implementations7 Dec 2023 Weilin Wan, Yiming Huang, Shutong Wu, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu

In this study, we introduce a learning-based method for generating high-quality human motion sequences from text descriptions (e. g., ``A person walks forward").

SMaRt: Improving GANs with Score Matching Regularity

no code implementations30 Nov 2023 Mengfei Xia, Yujun Shen, Ceyuan Yang, Ran Yi, Wenping Wang, Yong-Jin Liu

In this work, we revisit the mathematical foundations of GANs, and theoretically reveal that the native adversarial loss for GAN training is insufficient to fix the problem of subsets with positive Lebesgue measure of the generated data manifold lying out of the real data manifold.

valid

GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces

no code implementations29 Nov 2023 Yingwenqi Jiang, Jiadong Tu, YuAn Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, Yuexin Ma

In this paper, we present GaussianShader, a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces while preserving the training and rendering efficiency.

Neural Rendering

StructRe: Rewriting for Structured Shape Modeling

no code implementations29 Nov 2023 Jiepeng Wang, Hao Pan, Yang Liu, Xin Tong, Taku Komura, Wenping Wang

Such a localized rewriting process enables probabilistic modeling of ambiguous structures and robust generalization across object categories.

Object

TLControl: Trajectory and Language Control for Human Motion Synthesis

no code implementations28 Nov 2023 Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu

Controllable human motion synthesis is essential for applications in AR/VR, gaming, movies, and embodied AI.

Motion Synthesis

PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction

no code implementations20 Nov 2023 Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, Kai Zhang

We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing a 3D object from a few unposed images even with little visual overlap, while simultaneously estimating the relative camera poses in ~1. 3 seconds on a single A100 GPU.

3D Reconstruction Image to 3D +1

PERF: Panoramic Neural Radiance Field from a Single Panorama

1 code implementation25 Oct 2023 Guangcong Wang, Peng Wang, Zhaoxi Chen, Wenping Wang, Chen Change Loy, Ziwei Liu

In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.

Novel View Synthesis Text to 3D

Wonder3D: Single Image to 3D using Cross-Domain Diffusion

no code implementations23 Oct 2023 Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, YuAn Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, Wenping Wang

In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry.

Image to 3D

OAAFormer: Robust and Efficient Point Cloud Registration Through Overlapping-Aware Attention in Transformer

no code implementations15 Oct 2023 Junjie Gao, Qiujie Dong, Ruian Wang, Shuangmin Chen, Shiqing Xin, Changhe Tu, Wenping Wang

On one hand, we introduce a soft matching mechanism, facilitating the propagation of potentially valuable correspondences from coarse to fine levels.

Point Cloud Registration

Towards More Accurate Diffusion Model Acceleration with A Timestep Aligner

no code implementations14 Oct 2023 Mengfei Xia, Yujun Shen, Changsong Lei, Yu Zhou, Ran Yi, Deli Zhao, Wenping Wang, Yong-Jin Liu

By viewing the generation of diffusion models as a discretized integrating process, we argue that the quality drop is partly caused by applying an inaccurate integral direction to a timestep interval.

Denoising

MMPI: a Flexible Radiance Field Representation by Multiple Multi-plane Images Blending

no code implementations30 Sep 2023 Yuze He, Peng Wang, Yubin Hu, Wang Zhao, Ran Yi, Yong-Jin Liu, Wenping Wang

In this paper, we explore the potential of MPI and show that MPI can synthesize high-quality novel views of complex scenes with diverse camera distributions and view directions, which are not only limited to simple forward-facing scenes.

Autonomous Driving Novel View Synthesis

Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training

no code implementations29 Sep 2023 Runnan Chen, Xinge Zhu, Nenglun Chen, Dawei Wang, Wei Li, Yuexin Ma, Ruigang Yang, Tongliang Liu, Wenping Wang

In this paper, we propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.

3D Semantic Segmentation Object

C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters

no code implementations20 Sep 2023 Zhiyang Dou, Xuelin Chen, Qingnan Fan, Taku Komura, Wenping Wang

We present C$\cdot$ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for physics-based characters.

Imitation Learning

Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement

1 code implementation14 Sep 2023 Sheng Ye, Yubin Hu, Matthieu Lin, Yu-Hui Wen, Wang Zhao, Yong-Jin Liu, Wenping Wang

To enhance the normal priors, we introduce a simple yet effective image sharpening and denoising technique, coupled with a network that estimates the pixel-wise uncertainty of the predicted surface normal vectors.

Denoising Indoor Scene Reconstruction

SyncDreamer: Generating Multiview-consistent Images from a Single-view Image

2 code implementations7 Sep 2023 YuAn Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, Wenping Wang

In this paper, we present a novel diffusion model called that generates multiview-consistent images from a single-view image.

3D Generation Image to 3D +2

StyleAdapter: A Single-Pass LoRA-Free Model for Stylized Image Generation

no code implementations4 Sep 2023 Zhouxia Wang, Xintao Wang, Liangbin Xie, Zhongang Qi, Ying Shan, Wenping Wang, Ping Luo

StyleAdapter can generate high-quality images that match the content of the prompts and adopt the style of the references (even for unseen styles) in a single pass, which is more flexible and efficient than previous methods.

Image Generation

RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs

1 code implementation14 Aug 2023 Zhouxia Wang, Jiawei Zhang, Tianshui Chen, Wenping Wang, Ping Luo

In this work, we propose RestoreFormer++, which on the one hand introduces fully-spatial attention mechanisms to model the contextual information and the interplay with the priors, and on the other hand, explores an extending degrading model to help generate more realistic degraded face images to alleviate the synthetic-to-real-world gap.

Blind Face Restoration

Multi-Modal Machine Learning for Assessing Gaming Skills in Online Streaming: A Case Study with CS:GO

no code implementations23 Jul 2023 Longxiang Zhang, Wenping Wang

Moreover, we identify that our proposed models is prone to identifying users instead of learning meaningful representations.

Photo2Relief: Let Human in the Photograph Stand Out

no code implementations21 Jul 2023 Zhongping Ji, Feifei Che, Hanshuo Liu, Ziyi Zhao, Yu-Wei Zhang, Wenping Wang

The second challenge is that actual photographs often across different light conditions.

mCLIP: Multilingual CLIP via Cross-lingual Transfer

1 code implementation ACL 2023 Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang

Furthermore, to enhance the token- and sentence-level multilingual representation of the MTE, we propose to train it with machine translation and contrastive learning jointly before the TriKD to provide a better initialization.

Contrastive Learning Cross-Lingual Transfer +7

A Task-driven Network for Mesh Classification and Semantic Part Segmentation

no code implementations8 Jun 2023 Qiujie Dong, Xiaoran Gong, Rui Xu, Zixiong Wang, Shuangmin Chen, Shiqing Xin, Changhe Tu, Wenping Wang

With the rapid development of geometric deep learning techniques, many mesh-based convolutional operators have been proposed to bridge irregular mesh structures and popular backbone networks.

Segmentation Semantic Segmentation

NeuroGF: A Neural Representation for Fast Geodesic Distance and Path Queries

1 code implementation NeurIPS 2023 Qijian Zhang, Junhui Hou, Yohanes Yudhi Adikusuma, Wenping Wang, Ying He

To bridge this gap, this paper presents the first attempt to represent geodesics on 3D mesh models using neural implicit functions.

NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images

1 code implementation27 May 2023 YuAn Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, Wenping Wang

We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.

Neural Rendering Object

Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting

1 code implementation CVPR 2023 Haiping Wang, YuAn Liu, Zhen Dong, Yulan Guo, Yu-Shen Liu, Wenping Wang, Bisheng Yang

Previous multiview registration methods rely on exhaustive pairwise registration to construct a densely-connected pose graph and apply Iteratively Reweighted Least Square (IRLS) on the pose graph to compute the scan poses.

Point Cloud Registration

F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories

1 code implementation28 Mar 2023 Peng Wang, YuAn Liu, Zhaoxi Chen, Lingjie Liu, Ziwei Liu, Taku Komura, Christian Theobalt, Wenping Wang

Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.

Novel View Synthesis

NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing

no code implementations ICCV 2023 Zongcheng Li, Xiaoxiao Long, Yusen Wang, Tuo Cao, Wenping Wang, Fei Luo, Chunxia Xiao

In this paper, we propose to leverage implicit Signed Distance Function (SDF) as surface representation, and optimize the SDF field via volume rendering with a self-occlusion aware refractive ray tracing.

Transparent objects

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

1 code implementation CVPR 2023 Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang

For the first time, our pre-trained network achieves annotation-free 3D semantic segmentation with 20. 8% and 25. 08% mIoU on nuScenes and ScanNet, respectively.

3D Semantic Segmentation Contrastive Learning +4

F2-NeRF: Fast Neural Radiance Field Training With Free Camera Trajectories

no code implementations CVPR 2023 Peng Wang, YuAn Liu, Zhaoxi Chen, Lingjie Liu, Ziwei Liu, Taku Komura, Christian Theobalt, Wenping Wang

Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes.

Novel View Synthesis

SNAF: Sparse-view CBCT Reconstruction with Neural Attenuation Fields

no code implementations30 Nov 2022 Yu Fang, Lanzhuju Mei, Changjian Li, YuAn Liu, Wenping Wang, Zhiming Cui, Dinggang Shen

Cone beam computed tomography (CBCT) has been widely used in clinical practice, especially in dental clinics, while the radiation dose of X-rays when capturing has been a long concern in CBCT imaging.

GeoUDF: Surface Reconstruction from 3D Point Clouds via Geometry-guided Distance Representation

1 code implementation ICCV 2023 Siyu Ren, Junhui Hou, Xiaodong Chen, Ying He, Wenping Wang

We present a learning-based method, namely GeoUDF, to tackle the long-standing and challenging problem of reconstructing a discrete surface from a sparse point cloud. To be specific, we propose a geometry-guided learning method for UDF and its gradient estimation that explicitly formulates the unsigned distance of a query point as the learnable affine averaging of its distances to the tangent planes of neighboring points on the surface.

Surface Reconstruction

NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies

no code implementations CVPR 2023 Xiaoxiao Long, Cheng Lin, Lingjie Liu, YuAn Liu, Peng Wang, Christian Theobalt, Taku Komura, Wenping Wang

In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.

Neural Rendering

ToothInpaintor: Tooth Inpainting from Partial 3D Dental Model and 2D Panoramic Image

no code implementations25 Nov 2022 Yuezhi Yang, Zhiming Cui, Changjian Li, Wenping Wang

In this paper, we propose a neural network, called ToothInpaintor, that takes as input a partial 3D dental model and a 2D panoramic image and reconstructs the full tooth model with high-quality root(s).

An Implicit Parametric Morphable Dental Model

no code implementations21 Nov 2022 Congyi Zhang, Mohamed Elgharib, Gereon Fox, Min Gu, Christian Theobalt, Wenping Wang

Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum.

Batch-based Model Registration for Fast 3D Sherd Reconstruction

no code implementations ICCV 2023 Jiepeng Wang, Congyi Zhang, Peng Wang, Xin Li, Peter J. Cobb, Christian Theobalt, Wenping Wang

In this work, we aim to develop a portable, high-throughput, and accurate reconstruction system for efficient digitization of fragments excavated in archaeological sites.

3D Reconstruction

Zero-shot point cloud segmentation by transferring geometric primitives

no code implementations18 Oct 2022 Runnan Chen, Xinge Zhu, Nenglun Chen, Wei Li, Yuexin Ma, Ruigang Yang, Wenping Wang

To this end, we propose a novel framework to learn the geometric primitives shared in seen and unseen categories' objects and employ a fine-grained alignment between language and the learned geometric primitives.

Point Cloud Segmentation Semantic Segmentation

ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild

1 code implementation19 Jul 2022 Wang Zhao, Shaohui Liu, Hengkai Guo, Wenping Wang, Yong-Jin Liu

In addition, our method is able to retain reasonable accuracy of camera poses on fully static scenes, which consistently outperforms strong state-of-the-art dense correspondence based methods with end-to-end deep learning, demonstrating the potential of dense indirect methods based on optical flow and point trajectories.

Motion Segmentation Optical Flow Estimation +1

Progressively-connected Light Field Network for Efficient View Synthesis

no code implementations10 Jul 2022 Peng Wang, YuAn Liu, Guying Lin, Jiatao Gu, Lingjie Liu, Taku Komura, Wenping Wang

ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses.

Novel View Synthesis

NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors

no code implementations27 Jun 2022 Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, Wenping Wang

The key idea of NeuRIS is to integrate estimated normal of indoor scenes as a prior in a neural rendering framework for reconstructing large texture-less shapes and, importantly, to do this in an adaptive manner to also enable the reconstruction of irregular shapes with fine details.

3D Reconstruction Neural Rendering

Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions

no code implementations25 Jun 2022 Weilin Wan, Lei Yang, Lingjie Liu, Zhuoying Zhang, Ruixing Jia, Yi-King Choi, Jia Pan, Christian Theobalt, Taku Komura, Wenping Wang

We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.

Human-Object Interaction Detection motion prediction +1

SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views

1 code implementation12 Jun 2022 Xiaoxiao Long, Cheng Lin, Peng Wang, Taku Komura, Wenping Wang

We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images.

Neural Rendering Surface Reconstruction

Visual-Tactile Sensing for Real-time Liquid Volume Estimation in Grasping

no code implementations23 Feb 2022 Fan Zhu, Ruixing Jia, Lei Yang, Youcan Yan, Zheng Wang, Jia Pan, Wenping Wang

We propose a deep visuo-tactile model for realtime estimation of the liquid inside a deformable container in a proprioceptive way. We fuse two sensory modalities, i. e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor without any extra sensor calibrations. The robotic system is well controlled and adjusted based on the estimation model in real time.

Multi-Task Learning

Laplacian2Mesh: Laplacian-Based Mesh Understanding

1 code implementation1 Feb 2022 Qiujie Dong, Zixiong Wang, Manyi Li, Junjie Gao, Shuangmin Chen, Zhenyu Shu, Shiqing Xin, Changhe Tu, Wenping Wang

Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation.

Semantic Segmentation Surface Reconstruction

FaceFormer: Speech-Driven 3D Facial Animation with Transformers

1 code implementation CVPR 2022 Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, Taku Komura

Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data.

3D Face Animation

Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation

no code implementations4 Dec 2021 Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, Taku Komura

The existing datasets are collected to cover as many different phonemes as possible instead of sentences, thus limiting the capability of the audio-based model to learn more diverse contexts.

Language Modelling

Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation

1 code implementation16 Oct 2021 Guanhua Chen, Shuming Ma, Yun Chen, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

When applied to zero-shot cross-lingual abstractive summarization, it produces an average performance gain of 12. 3 ROUGE-L over mBART-ft. We conduct detailed analyses to understand the key ingredients of SixT+, including multilinguality of the auxiliary parallel data, positional disentangled encoder, and the cross-lingual transferability of its encoder.

Abstractive Text Summarization Cross-Lingual Abstractive Summarization +5

Referring Self-supervised Learning on 3D Point Cloud

no code implementations29 Sep 2021 Runnan Chen, Xinge Zhu, Nenglun Chen, Dawei Wang, Wei Li, Yuexin Ma, Ruigang Yang, Wenping Wang

In this paper, we study a new problem named Referring Self-supervised Learning (RSL) on 3D scene understanding: Given the 3D synthetic models with labels and the unlabeled 3D real scene scans, our goal is to distinguish the identical semantic objects on an unseen scene according to the referring synthetic 3D models.

Scene Understanding Self-Supervised Learning

PR-Net: Preference Reasoning for Personalized Video Highlight Detection

no code implementations ICCV 2021 Runnan Chen, Penghao Zhou, Wenzhe Wang, Nenglun Chen, Pai Peng, Xing Sun, Wenping Wang

Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention.

Highlight Detection Semantic Similarity +1

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

1 code implementation1 Sep 2021 Haiping Wang, YuAn Liu, Zhen Dong, Wenping Wang

In this paper, we propose a novel local descriptor-based framework, called You Only Hypothesize Once (YOHO), for the registration of two unaligned point clouds.

Ranked #5 on Point Cloud Registration on ETH (trained on 3DMatch) (Recall (30cm, 5 degrees) metric)

Point Cloud Registration

AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds

1 code implementation ICCV 2021 Runsong Zhu, YuAn Liu, Zhen Dong, Tengping Jiang, YuAn Wang, Wenping Wang, Bisheng Yang

Existing works use a network to learn point-wise weights for weighted least squares surface fitting to estimate the normals, which has difficulty in finding accurate normals in complex regions or containing noisy points.

Surface Normals Estimation

Neural Rays for Occlusion-aware Image-based Rendering

1 code implementation CVPR 2022 YuAn Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, Wenping Wang

On such a 3D point, these generalization methods will include inconsistent image features from invisible views, which interfere with the radiance field construction.

Neural Rendering Novel View Synthesis +1

DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation

1 code implementation27 Jul 2021 Yilin Wen, Xiangyu Li, Hao Pan, Lei Yang, Zheng Wang, Taku Komura, Wenping Wang

Scalable 6D pose estimation for rigid objects from RGB images aims at handling multiple objects and generalizing to novel objects.

6D Pose Estimation Metric Learning +2

Structure-Aware Long Short-Term Memory Network for 3D Cephalometric Landmark Detection

1 code implementation21 Jul 2021 Runnan Chen, Yuexin Ma, Nenglun Chen, Lingjie Liu, Zhiming Cui, Yanhong Lin, Wenping Wang

Detecting 3D landmarks on cone-beam computed tomography (CBCT) is crucial to assessing and quantifying the anatomical abnormalities in 3D cephalometric analysis.

Graph Attention regression

NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

6 code implementations NeurIPS 2021 Peng Wang, Lingjie Liu, YuAn Liu, Christian Theobalt, Taku Komura, Wenping Wang

In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.

Novel View Synthesis Surface Reconstruction

Semi-supervised Anatomical Landmark Detection via Shape-regulated Self-training

no code implementations28 May 2021 Runnan Chen, Yuexin Ma, Lingjie Liu, Nenglun Chen, Zhiming Cui, Guodong Wei, Wenping Wang

The global shape constraint is the inherent property of anatomical landmarks that provides valuable guidance for more consistent pseudo labelling of the unlabeled data, which is ignored in the previously semi-supervised methods.

Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective

no code implementations19 Apr 2021 Lei Chu, Hao Pan, Wenping Wang

We present a novel approach for completing and reconstructing 3D shapes from incomplete scanned data by using deep neural networks.

Category Disentangled Context: Turning Category-irrelevant Features Into Treasures

no code implementations1 Jan 2021 Keke Tang, Guodong Wei, Jie Zhu, Yuexin Ma, Runnan Chen, Zhaoquan Gu, Wenping Wang

Deep neural networks have achieved great success in computer vision, thanks to their ability in extracting category-relevant semantic features.

Image Classification

Learnable Motion Coherence for Correspondence Pruning

no code implementations CVPR 2021 YuAn Liu, Lingjie Liu, Cheng Lin, Zhen Dong, Wenping Wang

We propose a novel formulation of fitting coherent motions with a smooth function on a graph of correspondences and show that this formulation allows a closed-form solution by graph Laplacian.

Pose Estimation

Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks

1 code implementation CVPR 2021 Xiaoxiao Long, Lingjie Liu, Wei Li, Christian Theobalt, Wenping Wang

We present a novel method for multi-view depth estimation from a single video, which is a critical task in various applications, such as perception, reconstruction and robot navigation.

Depth Estimation Robot Navigation

SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform

1 code implementation22 Oct 2020 Cheng Lin, Lingjie Liu, Changjian Li, Leif Kobbelt, Bin Wang, Shiqing Xin, Wenping Wang

Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications.

Segmentation

Mapping in a cycle: Sinkhorn regularized unsupervised learning for point cloud shapes

1 code implementation ECCV 2020 Lei Yang, Wenxi Liu, Zhiming Cui, Nenglun Chen, Wenping Wang

We propose an unsupervised learning framework with the pretext task of finding dense correspondences between point cloud shapes from the same category based on the cycle-consistency formulation.

Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video

1 code implementation7 May 2020 Peng Wang, Lingjie Liu, Nenglun Chen, Hung-Kuo Chu, Christian Theobalt, Wenping Wang

We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.

Motion Estimation Occlusion Handling +1

MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera

no code implementations13 Apr 2020 Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu

We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning.

Occlusion-Aware Depth Estimation with Adaptive Normal Constraints

1 code implementation ECCV 2020 Xiaoxiao Long, Lingjie Liu, Christian Theobalt, Wenping Wang

We present a new learning-based method for multi-frame depth estimation from a color video, which is a fundamental problem in scene understanding, robot navigation or handheld 3D reconstruction.

3D Reconstruction Depth Estimation +2

Modeling 3D Shapes by Reinforcement Learning

2 code implementations ECCV 2020 Cheng Lin, Tingxiang Fan, Wenping Wang, Matthias Nießner

We explore how to enable machines to model 3D shapes like human modelers using deep reinforcement learning (RL).

Imitation Learning reinforcement-learning +1

Unsupervised Learning of Intrinsic Structural Representation Points

1 code implementation CVPR 2020 Nenglun Chen, Lingjie Liu, Zhiming Cui, Runnan Chen, Duygu Ceylan, Changhe Tu, Wenping Wang

The 3D structure points produced by our method encode the shape structure intrinsically and exhibit semantic consistency across all the shape instances with similar structures.

Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation

no code implementations14 Jan 2020 Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt

In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space.

Image-to-Image Translation Novel View Synthesis +1

Decision Propagation Networks for Image Classification

no code implementations27 Nov 2019 Keke Tang, Peng Song, Yuexin Ma, Zhaoquan Gu, Yu Su, Zhihong Tian, Wenping Wang

High-level (e. g., semantic) features encoded in the latter layers of convolutional neural networks are extensively exploited for image classification, leaving low-level (e. g., color) features in the early layers underexplored.

Classification General Classification +1

Cephalometric Landmark Detection by AttentiveFeature Pyramid Fusion and Regression-Voting

2 code implementations23 Aug 2019 Runnan Chen, Yuexin Ma, Nenglun Chen, Daniel Lee, Wenping Wang

Marking anatomical landmarks in cephalometric radiography is a critical operation in cephalometric analysis.

regression

Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans

no code implementations ICCV 2019 Cheng Lin, Changjian Li, Wenping Wang

We present a novel approach to align partial 3D reconstructions which may not have substantial overlap.

Attending Category Disentangled Global Context for Image Classification

no code implementations17 Dec 2018 Keke Tang, Guodong Wei, Runnan Chen, Jie Zhu, Zhaoquan Gu, Wenping Wang

In this paper, we propose a general framework for image classification using the attention mechanism and global context, which could incorporate with various network architectures to improve their performance.

Classification General Classification +1

TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents

1 code implementation6 Nov 2018 Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha

To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).

Autonomous Vehicles Navigate +2

Neural Rendering and Reenactment of Human Actor Videos

no code implementations11 Sep 2018 Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, Christian Theobalt

In contrast to conventional human character rendering, we do not require the availability of a production-quality photo-realistic 3D model of the human, but instead rely on a video sequence in conjunction with a (medium-quality) controllable 3D template model of the person.

Generative Adversarial Network Image Generation +1

Moiré Photo Restoration Using Multiresolution Convolutional Neural Networks

1 code implementation8 May 2018 Yujing Sun, Yizhou Yu, Wenping Wang

While digital image quality is constantly being improved, taking high-quality photos of digital screens still remains challenging because the photos are often contaminated with moir\'{e} patterns, a result of the interference between the pixel grids of the camera sensor and the device screen.

Denoising Image Enhancement +1

Efficient Reciprocal Collision Avoidance between Heterogeneous Agents Using CTMAT

no code implementations7 Apr 2018 Yuexin Ma, Dinesh Manocha, Wenping Wang

We present a novel algorithm for reciprocal collision avoidance between heterogeneous agents of different shapes and sizes.

Collision Avoidance

Deep Learning for Genomics: A Concise Overview

no code implementations2 Feb 2018 Tianwei Yue, Yuanxin Wang, Longxiang Zhang, Chunming Gu, Haoru Xue, Wenping Wang, Qi Lyu, Yujie Dun

Advancements in genomic research such as high-throughput sequencing techniques have driven modern genomic studies into "big data" disciplines.

A Sparse Graph-Structured Lasso Mixed Model for Genetic Association with Confounding Correction

1 code implementation11 Nov 2017 Wenting Ye, Xiang Liu, Tianwei Yue, Wenping Wang

We proposed the sparse graph-structured linear mixed model (sGLMM) that can incorporate the relatedness information from traits in a dataset with confounding correction.

Deep Multimodal Speaker Naming

no code implementations17 Jul 2015 Yongtao Hu, Jimmy Ren, Jingwen Dai, Chang Yuan, Li Xu, Wenping Wang

Automatic speaker naming is the problem of localizing as well as identifying each speaking character in a TV/movie/live show video.

Face Alignment

Cannot find the paper you are looking for? You can Submit a new open access paper.