Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds

13 Aug 2021  ·  Björn Michele, Alexandre Boulch, Gilles Puy, Maxime Bucher, Renaud Marlet ·

While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images, its application to 3D data is still recent and scarce, with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data, that can handle both classification and, for the first time, semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation, we created three benchmarks for evaluating this new ZSL task, using S3DIS, ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines, which we additionally propose for this task.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Generalized Zero-Shot Learning S3DIS 3DGenZ HmIoU 12.9 # 1
Generalized Zero-Shot Learning ScanNet 3DGenZ HmIoU 12.5 # 1
Generalized Zero-Shot Learning SemanticKITTI 3DGenZ HmIoU 17.1 # 1

Methods


No methods listed for this paper. Add relevant methods here