Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Data

18 Nov 2020  ·  Christoph B. Rist, David Emmerichs, Markus Enzweiler, Dariu M. Gavrila ·

Semantic scene completion is the task of jointly estimating 3D geometry and semantics of objects and surfaces within a given extent. This is a particularly challenging task on real-world data that is sparse and occluded. We propose a scene segmentation network based on local Deep Implicit Functions as a novel learning-based method for scene completion. Unlike previous work on scene completion, our method produces a continuous scene representation that is not based on voxelization. We encode raw point clouds into a latent space locally and at multiple spatial resolutions. A global scene completion function is subsequently assembled from the localized function patches. We show that this continuous representation is suitable to encode geometric and semantic properties of extensive outdoor scenes without the need for spatial discretization (thus avoiding the trade-off between level of scene detail and the scene extent that can be covered). We train and evaluate our method on semantically annotated LiDAR scans from the Semantic KITTI dataset. Our experiments verify that our method generates a powerful representation that can be decoded into a dense 3D description of a given scene. The performance of our method surpasses the state of the art on the Semantic KITTI Scene Completion Benchmark in terms of geometric completion intersection-over-union (IoU).

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Semantic Scene Completion SemanticKITTI Local-DIFs mIoU 22.7 # 4


No methods listed for this paper. Add relevant methods here