Index Network

11 Aug 2019  ·  Hao Lu, Yutong Dai, Chunhua Shen, Songcen Xu ·

We show that existing upsampling operators can be unified using the notion of the index function. This notion is inspired by an observation in the decoding process of deep image matting where indices-guided unpooling can often recover boundary details considerably better than other upsampling operators such as bilinear interpolation. By viewing the indices as a function of the feature map, we introduce the concept of "learning to index", and present a novel index-guided encoder-decoder framework where indices are self-learned adaptively from data and are used to guide the downsampling and upsampling stages, without extra training supervision. At the core of this framework is a new learnable module, termed Index Network (IndexNet), which dynamically generates indices conditioned on the feature map itself. IndexNet can be used as a plug-in applying to almost all off-the-shelf convolutional networks that have coupled downsampling and upsampling stages, giving the networks the ability to dynamically capture variations of local patterns. In particular, we instantiate and investigate five families of IndexNet and demonstrate their effectiveness on four dense prediction tasks, including image denoising, image matting, semantic segmentation, and monocular depth estimation. Code and models have been made available at:

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Grayscale Image Denoising BSD68 sigma15 Index Network PSNR 31.23 # 15
Grayscale Image Denoising BSD68 sigma25 Index Network PSNR 29.06 # 14
Grayscale Image Denoising BSD68 sigma50 Index Network PSNR 26.34 # 12
Monocular Depth Estimation NYU-Depth V2 Index Network RMSE 0.565 # 56
Grayscale Image Denoising Set12 sigma15 Index Network PSNR 32.82 # 5
Grayscale Image Denoising Set12 sigma30 Index Network PSNR 30.43 # 2
Grayscale Image Denoising Set12 sigma50 Index Network PSNR 27.29 # 6
Scene Segmentation SUN-RGBD Index Network Mean IoU 33.48 # 2