Semi-Supervised Semantic Segmentation with High- and Low-level Consistency

15 Aug 2019  ·  Sudhanshu Mittal, Maxim Tatarchenko, Thomas Brox ·

The ability to understand visual information from limited labeled data is an important aspect of machine learning. While image-level classification has been extensively studied in a semi-supervised setting, dense pixel-level classification with limited data has only drawn attention recently... In this work, we propose an approach for semi-supervised semantic segmentation that learns from limited pixel-wise annotated samples while exploiting additional annotation-free images. It uses two network branches that link semi-supervised classification with semi-supervised segmentation including self-training. The dual-branch approach reduces both the low-level and the high-level artifacts typical when training with few labels. The approach attains significant improvement over existing methods, especially when trained with very few labeled samples. On several standard benchmarks - PASCAL VOC 2012, PASCAL-Context, and Cityscapes - the approach achieves new state-of-the-art in semi-supervised learning. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled s4GAN (DeepLab v2 ImageNet pre-trained) Validation mIoU 59.3% # 13
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled s4GAN (DeepLab v2 ImageNet pre-trained) Validation mIoU 61.9% # 12
Semi-Supervised Semantic Segmentation Cityscapes 2% labeled S4GAN (DeepLabv2 with ResNet101, MSCOCO pre-trained) Validation mIoU 50.48% # 3
Semi-Supervised Semantic Segmentation Cityscapes 5% labeled S4GAN (DeepLabv2 with ResNet101, MSCOCO pre-trained) Validation mIoU 55.61% # 3
Semi-Supervised Semantic Segmentation PASCAL Context 12.5% labeled s4GAN+MLMT (DeepLab v2 ImageNet pre-trained) Validation mIoU 35.3 # 2
Semi-Supervised Semantic Segmentation PASCAL Context 25% labeled s4GAN+MLMT (DeepLab v2 ImageNet pre-trained) Validation mIoU 37.8 # 2
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled s4GAN+MLMT (DeepLab v2 ImageNet pre-trained) Validation mIoU 67.3% # 16
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled s4GAN + MLMT (DeepLab v2 MSCOCO/ImageNet pre-trained) Validation mIoU 71.4% # 10
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled s4GAN+MLMT (DeepLab v3 ImageNet pre-trained) Validation mIoU 70.4% # 14
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled s4GAN+MLMT (DeepLab v2 ImageNet pre-trained) Validation mIoU 60.4% # 11
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled s4GAN+MLMT (DeepLab v3+ ImageNet pre-trained) Validation mIoU 62.6% # 10
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled s4GAN + MLMT (DeepLab v2 MSCOCO/ImageNet pre-trained) Validation mIoU 63.3% # 9
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled s4GAN+MLMT (DeepLab v2 ImageNet pre-trained) Validation mIoU 62.9% # 11
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled s4GAN+MLMT (DeepLab v3+ ImageNet pre-trained) Validation mIoU 66.6% # 9
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled s4GAN + MLMT (DeepLab v2 MSCOCO/ImageNet pre-trained) Validation mIoU 67.2% # 8

Methods


No methods listed for this paper. Add relevant methods here