Semi-supervised semantic segmentation needs strong, varied perturbations

5 Jun 2019  ·  Geoff French, Samuli Laine, Timo Aila, Michal Mackiewicz, Graham Finlayson ·

Consistency regularization describes a class of approaches that have yielded ground breaking results in semi-supervised classification problems. Prior work has established the cluster assumption - under which the data distribution consists of uniform class clusters of samples separated by low density regions - as important to its success. We analyze the problem of semantic segmentation and find that its' distribution does not exhibit low density regions separating classes and offer this as an explanation for why semi-supervised segmentation is a challenging problem, with only a few reports of success. We then identify choice of augmentation as key to obtaining reliable performance without such low-density regions. We find that adapted variants of the recently proposed CutOut and CutMix augmentation techniques yield state-of-the-art semi-supervised semantic segmentation results in standard datasets. Furthermore, given its challenging nature we propose that semantic segmentation acts as an effective acid test for evaluating semi-supervised regularizers. Implementation at: https://github.com/Britefury/cutmix-semisup-seg.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled CutMix (DeepLab v2, ImageNet pre-trained) Validation mIoU 51.2 # 10
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled CutMix (DeepLab v2, ImageNet pre-trained) Validation mIoU 60.34% # 16
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled CutMix (DeepLab v2, ImageNet pre-trained) Validation mIoU 63.87% # 14
Semi-Supervised Semantic Segmentation nuScenes CutMix-Seg (Range View) mIoU (1% Labels) 43.8 # 5
mIoU (10% Labels) 63.9 # 5
mIoU (20% Labels) 64.8 # 7
mIoU (50% Labels) 69.8 # 5
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled CutMix Validation mIoU 67.6% # 19
Validation mIoU 72.45% # 12
Semi-Supervised Semantic Segmentation Pascal VOC 2012 1% labeled CutMix (DeepLab v3+ ImageNet pre-trained) Validation mIoU 59.52% # 4
Semi-Supervised Semantic Segmentation Pascal VOC 2012 1% labeled CutMix (DeepLab v2 ImageNet pre-trained) Validation mIoU 53.79% # 6
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled CutMix (DeepLab v2 ImageNet pre-trained) Validation mIoU 64.81% # 8
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled CutMix (DeepLab v3+ ImageNet pre-trained) Validation mIoU 67.05% # 5
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled CutMix (DeepLab v3+ ImageNet pre-trained) Validation mIoU 69.57% # 6
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled CutMix (DeepLab v2 ImageNet pre-trained) Validation mIoU 66.48% # 12
Semi-Supervised Semantic Segmentation ScribbleKITTI CutMix-Seg (Range View) mIoU (1% Labels) 36.7 # 5
mIoU (10% Labels) 50.7 # 3
mIoU (20% Labels) 52.9 # 3
mIoU (50% Labels) 54.3 # 5
Semi-Supervised Semantic Segmentation SemanticKITTI CutMix-Seg (Range View) mIoU (1% Labels) 37.4 # 7
mIoU (10% Labels) 54.3 # 5
mIoU (20% Labels) 56.6 # 4
mIoU (50% Labels) 57.6 # 5

Methods