Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

6 Nov 2018  ·  Xinhai Liu, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker ·

Exploring contextual information in the local region is important for shape understanding and analysis. Existing studies often employ hand-crafted or explicit ways to encode contextual information of local regions. However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features. To resolve this issue, we propose a novel deep learning model for 3D point clouds, named Point2Sequence, to learn 3D shape features by capturing fine-grained contextual information in a novel implicit way. Point2Sequence employs a novel sequence learning model for point clouds to capture the correlations by aggregating multi-scale areas of each local region with attention. Specifically, Point2Sequence first learns the feature of each area scale in a local region. Then, it captures the correlation between area scales in the process of aggregating all area scales using a recurrent neural network (RNN) based encoder-decoder structure, where an attention mechanism is proposed to highlight the importance of different area scales. Experimental results show that Point2Sequence achieves state-of-the-art performance in shape classification and segmentation tasks.

PDF Abstract

Datasets


Results from the Paper


Ranked #32 on 3D Part Segmentation on ShapeNet-Part (Instance Average IoU metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Point Cloud Classification ModelNet40 P2Sequence Overall Accuracy 92.6 # 41
3D Part Segmentation ShapeNet-Part P2Sequence Instance Average IoU 85.2 # 32

Methods


No methods listed for this paper. Add relevant methods here