Search Results for author: Zhizhong Han

Found 58 papers, 36 papers with code

Learning Continuous Implicit Field with Local Distance Indicator for Arbitrary-Scale Point Cloud Upsampling

no code implementations23 Dec 2023 Shujuan Li, Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Zhizhong Han

At inference time, we randomly sample queries around the sparse point cloud, and project these query points onto the zero-level set of the learned implicit field to generate a dense point cloud.

Differentiable Registration of Images and LiDAR Point Clouds with VoxelPoint-to-Pixel Matching

1 code implementation NeurIPS 2023 Junsheng Zhou, Baorui Ma, Wenyuan Zhang, Yi Fang, Yu-Shen Liu, Zhizhong Han

To address these problems, we propose to learn a structured cross-modality latent space to represent pixel features and 3D features via a differentiable probabilistic PnP solver.

NeuralGF: Unsupervised Point Normal Estimation by Learning Neural Gradient Function

1 code implementation NeurIPS 2023 Qing Li, Huifang Feng, Kanle Shi, Yue Gao, Yi Fang, Yu-Shen Liu, Zhizhong Han

Specifically, we introduce loss functions to facilitate query points to iteratively reach the moving targets and aggregate onto the approximated surface, thereby learning a global surface representation of the data.

Learning Neural Implicit through Volume Rendering with Attentive Depth Fusion Priors

no code implementations NeurIPS 2023 Pengchong Hu, Zhizhong Han

By introducing a novel attention mechanism, we allow neural networks to directly use the depth fusion prior with the inferred occupancy as the learned implicit function.

3D Reconstruction Simultaneous Localization and Mapping

Neural Gradient Learning and Optimization for Oriented Point Normal Estimation

1 code implementation17 Sep 2023 Qing Li, Huifang Feng, Kanle Shi, Yi Fang, Yu-Shen Liu, Zhizhong Han

We propose Neural Gradient Learning (NGL), a deep learning approach to learn gradient vectors with consistent orientation from 3D point clouds for normal estimation.

GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds

1 code implementation ICCV 2023 Chao Chen, Yu-Shen Liu, Zhizhong Han

However, these methods suffer from a slow inference due to the slow convergence of neural networks and the extensive calculation of distances to surface points, which limits them to small scale points.

Surface Reconstruction

Learning a More Continuous Zero Level Set in Unsigned Distance Fields through Level Set Projection

2 code implementations ICCV 2023 Junsheng Zhou, Baorui Ma, Shujuan Li, Yu-Shen Liu, Zhizhong Han

We pull the non-zero level sets onto the zero level set with gradient constraints which align gradients over different level sets and correct unsigned distance errors on the zero level set, leading to a smoother and more continuous unsigned distance field.

Surface Reconstruction

Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

1 code implementation ICCV 2023 Sijia Jiang, Jing Hua, Zhizhong Han

To resolve this issue in a general sense, we introduce to learn neural implicit representations with quantized coordinates, which reduces the uncertainty and ambiguity in the field during optimization.

3D Reconstruction

Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping

1 code implementation2 Jun 2023 Baorui Ma, Yu-Shen Liu, Zhizhong Han

However, without ground truth signed distances, point normals or clean point clouds, current methods still struggle from learning SDFs from noisy point clouds.

Denoising Surface Reconstruction

Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignment

1 code implementation CVPR 2023 Baorui Ma, Junsheng Zhou, Yu-Shen Liu, Zhizhong Han

Our insight is to propagate the zero level set to everywhere in the field through consistent gradients to eliminate uncertainty in the field that is caused by the discreteness of 3D point clouds or the lack of observations from multi-view images.

SHS-Net: Learning Signed Hyper Surfaces for Oriented Normal Estimation of Point Clouds

1 code implementation CVPR 2023 Qing Li, Huifang Feng, Kanle Shi, Yue Gao, Yi Fang, Yu-Shen Liu, Zhizhong Han

In this work, we introduce signed hyper surfaces (SHS), which are parameterized by multi-layer perceptron (MLP) layers, to learn to estimate oriented normals from point clouds in an end-to-end manner.

D-Net: Learning for Distinctive Point Clouds by Self-Attentive Point Searching and Learnable Feature Fusion

no code implementations10 May 2023 Xinhai Liu, Zhizhong Han, Sanghuk Lee, Yan-Pei Cao, Yu-Shen Liu

Most of early methods selected the important points on 3D shapes by analyzing the intrinsic geometric properties of every single shape, which fails to capture the importance of points that distinguishes a shape from objects of other classes, i. e., the distinction of points.

Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priors

1 code implementation CVPR 2023 Chao Chen, Yu-Shen Liu, Zhizhong Han

To resolve this issue, we present a neural network to directly infer SDFs from single sparse point clouds without using signed distance supervision, learned priors or even normals.

Surface Reconstruction

LP-DIF: Learning Local Pattern-Specific Deep Implicit Function for 3D Objects and Scenes

no code implementations CVPR 2023 Meng Wang, Yu-Shen Liu, Yue Gao, Kanle Shi, Yi Fang, Zhizhong Han

To capture geometry details, current mainstream methods divide 3D shapes into local regions and then learn each one with a local latent code via a decoder, where the decoder shares the geometric similarities among different local regions.

3D Reconstruction 3D Shape Representation

NeAF: Learning Neural Angle Fields for Point Normal Estimation

1 code implementation30 Nov 2022 Shujuan Li, Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Zhizhong Han

To resolve these issues, we propose an implicit function to learn an angle field around the normal of each point in the spherical coordinate system, which is dubbed as Neural Angle Fields (NeAF).

Surface Normals Estimation

HSurf-Net: Normal Estimation for 3D Point Clouds by Learning Hyper Surfaces

1 code implementation13 Oct 2022 Qing Li, Yu-Shen Liu, Jin-San Cheng, Cheng Wang, Yi Fang, Zhizhong Han

To address these issues, we introduce hyper surface fitting to implicitly learn hyper surfaces, which are represented by multi-layer perceptron (MLP) layers that take point features as input and output surface patterns in a high dimensional feature space.

Surface Normals Estimation

Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds

1 code implementation6 Oct 2022 Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Yi Fang, Zhizhong Han

In this paper, we propose a novel method to learn consistency-aware unsigned distance functions directly from raw point clouds.

Surface Reconstruction

Fast Learning Radiance Fields by Shooting Much Fewer Rays

1 code implementation14 Aug 2022 Wenyuan Zhang, Ruofan Xing, Yunfan Zeng, Yu-Shen Liu, Kanle Shi, Zhizhong Han

Experimental results show that our method achieves comparable accuracy to the state-of-the-art with much faster training.

Novel View Synthesis

Latent Partition Implicit with Surface Codes for 3D Representation

1 code implementation18 Jul 2022 Chao Chen, Yu-Shen Liu, Zhizhong Han

Our insight here is that both the part learning and the part blending can be conducted much easier in the latent space than in the spatial space.

Unity

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

1 code implementation CVPR 2022 Baorui Ma, Yu-Shen Liu, Matthias Zwicker, Zhizhong Han

To reconstruct a surface at a specific query location at inference time, these methods then match the local reconstruction target by searching for the best match in the local prior space (by optimizing the parameters encoding the local context) at the given query location.

Surface Reconstruction

Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors

2 code implementations CVPR 2022 Baorui Ma, Yu-Shen Liu, Zhizhong Han

Our key idea is to infer signed distances by pushing both the query projections to be on the surface and the projection distance to be the minimum.

3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow

1 code implementation CVPR 2022 Xin Wen, Junsheng Zhou, Yu-Shen Liu, Zhen Dong, Zhizhong Han

Reconstructing 3D shape from a single 2D image is a challenging task, which needs to estimate the detailed 3D structures based on the semantic attributes from 2D image.

3D Reconstruction 3D Shape Reconstruction +1

Learning Deep Implicit Functions for 3D Shapes with Dynamic Code Clouds

1 code implementation CVPR 2022 Tianyang Li, Xin Wen, Yu-Shen Liu, Hua Su, Zhizhong Han

However, the local codes are constrained at discrete and regular positions like grid points, which makes the code positions difficult to be optimized and limits their representation ability.

3D Shape Representation Position

3D-OAE: Occlusion Auto-Encoders for Self-Supervised Learning on Point Clouds

1 code implementation26 Mar 2022 Junsheng Zhou, Xin Wen, Baorui Ma, Yu-Shen Liu, Yue Gao, Yi Fang, Zhizhong Han

To address this problem, we present a novel and efficient self-supervised point cloud representation learning framework, named 3D Occlusion Auto-Encoder (3D-OAE), to facilitate the detailed supervision inherited in local regions and global shapes.

Representation Learning Self-Supervised Learning

PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Paths

1 code implementation19 Feb 2022 Xin Wen, Peng Xiang, Zhizhong Han, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Yu-Shen Liu

It moves each point of incomplete input to obtain a complete point cloud, where total distance of point moving paths (PMPs) should be the shortest.

Point Cloud Completion Representation Learning

Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer

1 code implementation18 Feb 2022 Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Zhizhong Han

Our insight into the detailed geometry is to introduce a skip-transformer in the SPD to learn the point splitting patterns that can best fit the local regions.

Image Reconstruction Point Cloud Completion

SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer

2 code implementations ICCV 2021 Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Zhizhong Han

However, previous methods usually suffered from discrete nature of point cloud and unstructured prediction of points in local regions, which makes it hard to reveal fine local geometric details on the complete shape.

Point Cloud Completion

Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching

1 code implementation8 Aug 2021 Chen Chao, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Our method pushes the neural network to generate a 3D point cloud whose 2D projections match the irregular point supervision from different view angles.

Point Cloud Generation

Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views

no code implementations8 Aug 2021 Zhizhong Han, Xiyang Wang, Yu-Shen Liu, Matthias Zwicker

To mine highly discriminative information from unordered views, HVP performs a novel hierarchical view prediction over a view pair, and aggregates the knowledge learned from the predictions in all view pairs into a global feature.

Retrieval

Parts2Words: Learning Joint Embedding of Point Clouds and Texts by Bidirectional Matching between Parts and Words

1 code implementation CVPR 2023 Chuan Tang, Xi Yang, Bojian Wu, Zhizhong Han, Yi Chang

Specifically, we first segment the point clouds into parts, and then leverage optimal transport method to match parts and words in an optimized feature space, where each part is represented by aggregating features of all points within it and each word is abstracted by its contextual information.

Retrieval Text Matching

Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Coding

1 code implementation CVPR 2021 Xin Wen, Zhizhong Han, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Yu-Shen Liu

We provide a comprehensive evaluation in experiments, which shows that our model with the learned bidirectional geometry correspondence outperforms state-of-the-art unpaired completion methods.

Point Cloud Completion

Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projections Matching

1 code implementation ICCV 2021 Chao Chen, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Our method pushes the neural network to generate a 3D point cloud whose 2D projections match the irregular point supervision from different view angles.

Point Cloud Generation

SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization

1 code implementation8 Dec 2020 Xinhai Liu, Xinchen Liu, Yu-Shen Liu, Zhizhong Han

The task of point cloud upsampling aims to acquire dense and uniform point sets from sparse and irregular point sets.

PMP-Net: Point Cloud Completion by Learning Multi-step Point Moving Paths

1 code implementation CVPR 2021 Xin Wen, Peng Xiang, Zhizhong Han, Yan-Pei Cao, Pengfei Wan, Wen Zheng, Yu-Shen Liu

As a result, the network learns a strict and unique correspondence on point-level, which can capture the detailed topology and structure relationships between the incomplete shape and the complete target, and thus improves the quality of the predicted complete shape.

Point Cloud Completion

Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces

1 code implementation26 Nov 2020 Baorui Ma, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

Specifically, we train a neural network to pull query 3D locations to their closest points on the surface using the predicted signed distance values and the gradient at the query locations, both of which are computed by the network itself.

Image Reconstruction Surface Reconstruction

DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images

no code implementations ICML 2020 Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker

To optimize 3D shape parameters, current renderers rely on pixel-wise losses between rendered images of 3D reconstructions and ground truth images from corresponding viewpoints.

Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions

1 code implementation26 May 2020 Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

According to our experiments under this fine-grained dataset, we find that state-of-the-art methods are significantly limited by the small variance among subcategories in the same category.

3D Shape Classification General Classification +2

Point Cloud Completion by Skip-attention Network with Hierarchical Folding

no code implementations CVPR 2020 Xin Wen, Tianyang Li, Zhizhong Han, Yu-Shen Liu

Point cloud completion aims to infer the complete geometries for missing regions of 3D objects from incomplete ones.

Point Cloud Completion

LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts

no code implementations18 Mar 2020 Xinhai Liu, Zhizhong Han, Fangzhou Hong, Yu-Shen Liu, Matthias Zwicker

However, due to the irregularity and sparsity in sampled point clouds, it is hard to encode the fine-grained geometry of local regions and their spatial relationships when only using the fixed-size filters and individual local feature integration, which limit the ability to learn discriminative features.

SeqXY2SeqZ: Structure Learning for 3D Shapes by Sequentially Predicting 1D Occupancy Segments From 2D Coordinates

no code implementations ECCV 2020 Zhizhong Han, Guanhui Qiao, Yu-Shen Liu, Matthias Zwicker

To avoid dense and irregular sampling in 3D, we propose to represent shapes using 2D functions, where the output of the function at each 2D location is a sequence of line segments inside the shape.

Learning to Generate Dense Point Clouds with Textures on Multiple Categories

1 code implementation22 Dec 2019 Tao Hu, Geng Lin, Zhizhong Han, Matthias Zwicker

In this paper, we propose a novel approach for reconstructing point clouds from RGB images.

3D Reconstruction

SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization

1 code implementation CVPR 2020 Yue Jiang, Dantong Ji, Zhizhong Han, Matthias Zwicker

We propose SDFDiff, a novel approach for image-based shape optimization using differentiable rendering of 3D shapes represented by signed distance functions (SDFs).

 Ranked #1 on Single-View 3D Reconstruction on ShapeNet (using extra training data)

3D Reconstruction Multi-View 3D Reconstruction +1

3D Shape Completion with Multi-view Consistent Inference

no code implementations28 Nov 2019 Tao Hu, Zhizhong Han, Matthias Zwicker

We formulate the regularization term as a consistency loss that encourages geometric consistency among multiple views, while the data term guarantees that the optimized views do not drift away too much from a learned shape descriptor.

Fast Low-rank Metric Learning for Large-scale and High-dimensional Data

1 code implementation NeurIPS 2019 Han Liu, Zhizhong Han, Yu-Shen Liu, Ming Gu

Low-rank metric learning aims to learn better discrimination of data subject to low-rank constraints.

Metric Learning

Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules

no code implementations29 Aug 2019 Xin Wen, Zhizhong Han, Xinhai Liu, Yu-Shen Liu

Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters.

L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention

no code implementations2 Aug 2019 Xinhai Liu, Zhizhong Han, Xin Wen, Yu-Shen Liu, Matthias Zwicker

Specifically, L2G-AE employs an encoder to encode the geometry information of multiple scales in a local region at the same time.

Retrieval

ShapeCaptioner: Generative Caption Network for 3D Shapes by Learning a Mapping from Parts Detected in Multiple Views to Sentences

no code implementations31 Jul 2019 Zhizhong Han, Chao Chen, Yu-Shen Liu, Matthias Zwicker

Specifically, ShapeCaptioner aggregates the parts detected in multiple colored views using our novel part class specific aggregation to represent a 3D shape, and then, employs a sequence to sequence model to generate the caption.

Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views

no code implementations18 May 2019 Zhizhong Han, Xinhai Liu, Yu-Shen Liu, Matthias Zwicker

In contrast, we propose a deep neural network, called Parts4Feature, to learn 3D global features from part-level information in multiple views.

Region Proposal

3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention

no code implementations17 May 2019 Zhizhong Han, Xiyang Wang, Chi-Man Vong, Yu-Shen Liu, Matthias Zwicker, C. L. Philip Chen

Then, the content and spatial information of each pair of view nodes are encoded by a novel spatial pattern correlation, where the correlation is computed among latent semantic patterns.

Render4Completion: Synthesizing Multi-View Depth Maps for 3D Shape Completion

no code implementations17 Apr 2019 Tao Hu, Zhizhong Han, Abhinav Shrivastava, Matthias Zwicker

Different from image-to-image translation network that completes each view separately, our novel network, multi-view completion net (MVCN), leverages information from all views of a 3D shape to help the completion of each single view.

Image-to-Image Translation Translation

Adversarial Cross-Modal Retrieval via Learning and Transferring Single-Modal Similarities

no code implementations17 Apr 2019 Xin Wen, Zhizhong Han, Xinyu Yin, Yu-Shen Liu

Cross-modal retrieval aims to retrieve relevant data across different modalities (e. g., texts vs. images).

Cross-Modal Retrieval Retrieval

Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences

no code implementations7 Nov 2018 Zhizhong Han, Mingyang Shang, Xiyang Wang, Yu-Shen Liu, Matthias Zwicker

A recent method employs 3D voxels to represent 3D shapes, but this limits the approach to low resolutions due to the computational cost caused by the cubic complexity of 3D voxels.

3D Shape Representation Cross-Modal Retrieval +2

View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions

no code implementations7 Nov 2018 Zhizhong Han, Mingyang Shang, Yu-Shen Liu, Matthias Zwicker

Intuitively, this memory enables the system to aggregate information that is useful to better solve the view inter-prediction tasks for each shape, and to leverage the memory as a view-independent shape representation.

3D Point Cloud Linear Classification Representation Learning

Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

no code implementations6 Nov 2018 Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker

However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features.

3D Part Segmentation 3D Point Cloud Classification +1

Cannot find the paper you are looking for? You can Submit a new open access paper.