Sparsity Invariant CNNs

22 Aug 2017  ·  Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, Andreas Geiger ·

In this paper, we consider convolutional neural networks operating on sparse inputs with an application to depth upsampling from sparse laser scan data. First, we show that traditional convolutional networks perform poorly when applied to sparse data even when the location of missing data is provided to the network. To overcome this problem, we propose a simple yet effective sparse convolution layer which explicitly considers the location of missing data during the convolution operation. We demonstrate the benefits of the proposed network architecture in synthetic and real experiments with respect to various baseline approaches. Compared to dense baselines, the proposed sparse convolution network generalizes well to novel datasets and is invariant to the level of sparsity in the data. For our evaluation, we derive a novel dataset from the KITTI benchmark, comprising 93k depth annotated RGB images. Our dataset allows for training and evaluating depth upsampling and depth prediction techniques in challenging real-world settings and will be made available upon publication.

PDF Abstract

Datasets


Introduced in the Paper:

KITTI-Depth

Used in the Paper:

Cityscapes KITTI SYNTHIA

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Depth Completion KITTI Depth Completion SparseConvs RMSE 1601 # 15
MAE 481 # 13
Runtime [ms] 10 # 1

Methods