Search Results for author:
Found 83 papers, 39 papers with code
TANet: Towards Fully Automatic Tooth Arrangement
Determining optimal target tooth arrangements is a key step of treatment planning in digital orthodontics.
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation
When applied to zero-shot cross-lingual abstractive summarization, it produces an average performance gain of 12. 3 ROUGE-L over mBART-ft. We conduct detailed analyses to understand the key ingredients of SixT+, including multilinguality of the auxiliary parallel data, positional disentangled encoder, and the cross-lingual transferability of its encoder.
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
We present C$\cdot$ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for physics-based characters.
Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement
To improve the capacity of the implicit representation, we propose a hybrid architecture to represent low-frequency and high-frequency regions separately.
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
In this paper, we present a novel diffusion model called that generates multiview-consistent images from a single-view image.
StyleAdapter: A Single-Pass LoRA-Free Model for Stylized Image Generation
StyleAdapter can generate high-quality images that match the content of the prompts and adopt the style of the references (even for unseen styles) in a single pass, which is more flexible and efficient than previous methods.
Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian
Our approach enforces the Hessian of the neural implicit function to have a zero determinant for points near the surface.
RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs
In this work, we propose RestoreFormer++, which on the one hand introduces fully-spatial attention mechanisms to model the contextual information and the interplay with the priors, and on the other hand, explores an extending degrading model to help generate more realistic degraded face images to alleviate the synthetic-to-real-world gap.
Multi-Modal Machine Learning for Assessing Gaming Skills in Online Streaming: A Case Study with CS:GO
Moreover, we identify that our proposed models is prone to identifying users instead of learning meaningful representations.
Photo2Relief: Let Human in the Photograph Stand Out
The second challenge is that actual photographs often across different light conditions.
Towards Label-free Scene Understanding by Vision Foundation Models
Vision foundation models such as Contrastive Vision-Language Pre-training (CLIP) and Segment Anything (SAM) have demonstrated impressive zero-shot performance on image classification and segmentation tasks.
NeuroGF: A Neural Representation for Fast Geodesic Distance and Path Queries
To bridge this gap, this paper presents the first attempt to represent geodesics on 3D mesh models using neural implicit functions.
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.
Integrity and Junkiness Failure Handling for Embedding-based Retrieval: A Case Study in Social Network Search
Embedding based retrieval has seen its usage in a variety of search applications like e-commerce, social networking search etc.
Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting
Previous multiview registration methods rely on exhaustive pairwise registration to construct a densely-connected pose graph and apply Iteratively Reweighted Least Square (IRLS) on the pose graph to compute the scan poses.
F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing
In this paper, we propose to leverage implicit Signed Distance Function (SDF) as surface representation, and optimize the SDF field via volume rendering with a self-occlusion aware refractive ray tracing.
CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
For the first time, our pre-trained network achieves annotation-free 3D semantic segmentation with 20. 8% and 25. 08% mIoU on nuScenes and ScanNet, respectively.
F2-NeRF: Fast Neural Radiance Field Training With Free Camera Trajectories
Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes.
GeoUDF: Surface Reconstruction from 3D Point Clouds via Geometry-guided Distance Representation
We present a learning-based method, namely GeoUDF, to tackle the long-standing and challenging problem of reconstructing a discrete surface from a sparse point cloud. To be specific, we propose a geometry-guided learning method for UDF and its gradient estimation that explicitly formulates the unsigned distance of a query point as the learnable affine averaging of its distances to the tangent planes of neighboring points on the surface.
SNAF: Sparse-view CBCT Reconstruction with Neural Attenuation Fields
Cone beam computed tomography (CBCT) has been widely used in clinical practice, especially in dental clinics, while the radiation dose of X-rays when capturing has been a long concern in CBCT imaging.
ToothInpaintor: Tooth Inpainting from Partial 3D Dental Model and 2D Panoramic Image
In this paper, we propose a neural network, called ToothInpaintor, that takes as input a partial 3D dental model and a 2D panoramic image and reconstructs the full tooth model with high-quality root(s).
NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies
In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
An Implicit Parametric Morphable Dental Model
Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum.
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
We further demonstrate the generalizability of our method on hand mesh recovery.
Batch-based Model Registration for Fast 3D Sherd Reconstruction
In this work, we aim to develop a portable, high-throughput, and accurate reconstruction system for efficient digitization of fragments excavated in archaeological sites.
Bridging Language and Geometric Primitives for Zero-shot Point Cloud Segmentation
To this end, we propose a novel framework to learn the geometric primitives shared in seen and unseen categories' objects and employ a fine-grained alignment between language and the learned geometric primitives.
Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos
Understanding dynamic hand motions and actions from egocentric RGB videos is a fundamental yet challenging task due to self-occlusion and ambiguity.
ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild
In addition, our method is able to retain reasonable accuracy of camera poses on fully static scenes, which consistently outperforms strong state-of-the-art dense correspondence based methods with end-to-end deep learning, demonstrating the potential of dense indirect methods based on optical flow and point trajectories.
Progressively-connected Light Field Network for Efficient View Synthesis
ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses.
NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors
The key idea of NeuRIS is to integrate estimated normal of indoor scenes as a prior in a neural rendering framework for reconstructing large texture-less shapes and, importantly, to do this in an adaptive manner to also enable the reconstruction of irregular shapes with fine details.
Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions
We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.
SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images.
Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
In this paper, we present a generalizable model-free 6-DoF object pose estimator called Gen6D.
Self-Supervised Image Representation Learning with Geometric Set Consistency
We propose a method for self-supervised image representation learning under the guidance of 3D geometric consistency.
Towards 3D Scene Understanding by Referring Synthetic Models
Promising performance has been achieved for visual perception on the point cloud.
Visual-Tactile Sensing for Real-time Liquid Volume Estimation in Grasping
We propose a deep visuo-tactile model for realtime estimation of the liquid inside a deformable container in a proprioceptive way. We fuse two sensory modalities, i. e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor without any extra sensor calibrations. The robotic system is well controlled and adjusted based on the estimation model in real time.
Laplacian2Mesh: Laplacian-Based Mesh Understanding
Geometric deep learning has sparked a rising interest in computer graphics to perform shape understanding tasks, such as shape classification and semantic segmentation.
RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs
Blind face restoration is to recover a high-quality face image from unknown degradations.
FaceFormer: Speech-Driven 3D Facial Animation with Transformers
Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data.
Ranked #1 on
3D Face Animation
on VOCASET
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation
The existing datasets are collected to cover as many different phonemes as possible instead of sentences, thus limiting the capability of the audio-based model to learn more diverse contexts.
Dense Representative Tooth Landmark/axis Detection Network on 3D Model
Results show that our method can produce tooth landmarks with high accuracy.
Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation
When applied to zero-shot cross-lingual abstractive summarization, it produces an average performance gain of 12. 3 ROUGE-L over mBART-ft. We conduct detailed analyses to understand the key ingredients of SixT+, including multilinguality of the auxiliary parallel data, positional disentangled encoder, and the cross-lingual transferability of its encoder.
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Referring Self-supervised Learning on 3D Point Cloud
In this paper, we study a new problem named Referring Self-supervised Learning (RSL) on 3D scene understanding: Given the 3D synthetic models with labels and the unlabeled 3D real scene scans, our goal is to distinguish the identical semantic objects on an unseen scene according to the referring synthetic 3D models.
Neural-IMLS: Self-supervised Implicit Moving Least-Squares Network for Surface Reconstruction
We conducted extensive experiments on various benchmarks, including synthetic scans and real scans.
PR-Net: Preference Reasoning for Personalized Video Highlight Detection
Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention.
You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors
In this paper, we propose a novel local descriptor-based framework, called You Only Hypothesize Once (YOHO), for the registration of two unaligned point clouds.
CODEs: Chamfer Out-of-Distribution Examples against Overconfidence Issue
Overconfident predictions on out-of-distribution (OOD) samples is a thorny issue for deep neural networks.
AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds
Existing works use a network to learn point-wise weights for weighted least squares surface fitting to estimate the normals, which has difficulty in finding accurate normals in complex regions or containing noisy points.
Ranked #5 on
Surface Normals Estimation
on PCPNet
Distributed Attention for Grounded Image Captioning
We study the problem of weakly supervised grounded image captioning.
Neural Rays for Occlusion-aware Image-based Rendering
On such a 3D point, these generalization methods will include inconsistent image features from invisible views, which interfere with the radiance field construction.
DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation
Scalable 6D pose estimation for rigid objects from RGB images aims at handling multiple objects and generalizing to novel objects.
Structure-Aware Long Short-Term Memory Network for 3D Cephalometric Landmark Detection
Detecting 3D landmarks on cone-beam computed tomography (CBCT) is crucial to assessing and quantifying the anatomical abnormalities in 3D cephalometric analysis.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.
Semi-supervised Anatomical Landmark Detection via Shape-regulated Self-training
The global shape constraint is the inherent property of anatomical landmarks that provides valuable guidance for more consistent pseudo labelling of the unlabeled data, which is ignored in the previously semi-supervised methods.
Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective
We present a novel approach for completing and reconstructing 3D shapes from incomplete scanned data by using deep neural networks.
Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
In this paper, we focus on a zero-shot cross-lingual transfer task in NMT.
Adaptive Surface Normal Constraint for Depth Estimation
We present a novel method for single image depth estimation using surface normal constraints.
Category Disentangled Context: Turning Category-irrelevant Features Into Treasures
Deep neural networks have achieved great success in computer vision, thanks to their ability in extracting category-relevant semantic features.
Point2Skeleton: Learning Skeletal Representations from Point Clouds
We introduce Point2Skeleton, an unsupervised method to learn skeletal representations from point clouds.
Learnable Motion Coherence for Correspondence Pruning
We propose a novel formulation of fitting coherent motions with a smooth function on a graph of correspondences and show that this formulation allows a closed-form solution by graph Laplacian.
Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks
We present a novel method for multi-view depth estimation from a single video, which is a critical task in various applications, such as perception, reconstruction and robot navigation.
SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform
Segmenting arbitrary 3D objects into constituent parts that are structurally meaningful is a fundamental problem encountered in a wide range of computer graphics applications.
Mapping in a cycle: Sinkhorn regularized unsupervised learning for point cloud shapes
We propose an unsupervised learning framework with the pretext task of finding dense correspondences between point cloud shapes from the same category based on the cycle-consistency formulation.
Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video
We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.
MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera
We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning.
Occlusion-Aware Depth Estimation with Adaptive Normal Constraints
We present a new learning-based method for multi-frame depth estimation from a color video, which is a fundamental problem in scene understanding, robot navigation or handheld 3D reconstruction.
Modeling 3D Shapes by Reinforcement Learning
We explore how to enable machines to model 3D shapes like human modelers using deep reinforcement learning (RL).
Unsupervised Learning of Intrinsic Structural Representation Points
The 3D structure points produced by our method encode the shape structure intrinsically and exhibit semantic consistency across all the shape instances with similar structures.
Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation
In this paper, we propose a novel human video synthesis method that approaches these limiting factors by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space.
Decision Propagation Networks for Image Classification
High-level (e. g., semantic) features encoded in the latter layers of convolutional neural networks are extensively exploited for image classification, leaving low-level (e. g., color) features in the early layers underexplored.
Cephalometric Landmark Detection by AttentiveFeature Pyramid Fusion and Regression-Voting
Marking anatomical landmarks in cephalometric radiography is a critical operation in cephalometric analysis.
Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans
We present a novel approach to align partial 3D reconstructions which may not have substantial overlap.
Attending Category Disentangled Global Context for Image Classification
In this paper, we propose a general framework for image classification using the attention mechanism and global context, which could incorporate with various network architectures to improve their performance.
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).
Ranked #1 on
Trajectory Prediction
on Apolloscape Trajectory
Neural Rendering and Reenactment of Human Actor Videos
In contrast to conventional human character rendering, we do not require the availability of a production-quality photo-realistic 3D model of the human, but instead rely on a video sequence in conjunction with a (medium-quality) controllable 3D template model of the person.
Moiré Photo Restoration Using Multiresolution Convolutional Neural Networks
While digital image quality is constantly being improved, taking high-quality photos of digital screens still remains challenging because the photos are often contaminated with moir\'{e} patterns, a result of the interference between the pixel grids of the camera sensor and the device screen.
Ranked #6 on
Image Enhancement
on TIP 2018
Efficient Reciprocal Collision Avoidance between Heterogeneous Agents Using CTMAT
We present a novel algorithm for reciprocal collision avoidance between heterogeneous agents of different shapes and sizes.
Deep Learning for Genomics: A Concise Overview
Advancements in genomic research such as high-throughput sequencing techniques have driven modern genomic studies into "big data" disciplines.
A Sparse Graph-Structured Lasso Mixed Model for Genetic Association with Confounding Correction
We proposed the sparse graph-structured linear mixed model (sGLMM) that can incorporate the relatedness information from traits in a dataset with confounding correction.
Harvesting Discriminative Meta Objects with Deep CNN Features for Scene Classification
Recent work on scene classification still makes use of generic CNN features in a rudimentary manner.
Deep Multimodal Speaker Naming
Automatic speaker naming is the problem of localizing as well as identifying each speaking character in a TV/movie/live show video.
SCaLE: Supervised and Cascaded Laplacian Eigenmaps for Visual Object Recognition Based on Nearest Neighbors
In this paper we develop a novel deep learning method that facilitates examplebased visual object category recognition.
