Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bank

This work presents a novel approach for semi-supervised semantic segmentation. The key element of this approach is our contrastive learning module that enforces the segmentation network to yield similar pixel-level feature representations for same-class samples across the whole dataset. To achieve this, we maintain a memory bank continuously updated with relevant and high-quality feature vectors from labeled data. In an end-to-end training, the features from both labeled and unlabeled data are optimized to be similar to same-class samples from the memory bank. Our approach outperforms the current state-of-the-art for semi-supervised semantic segmentation and semi-supervised domain adaptation on well-known public benchmarks, with larger improvements on the most challenging scenarios, i.e., less available labeled data. https://github.com/Shathe/SemiSeg-Contrastive

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 59.4% # 4
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled SemiSegContrast (DeepLab v3+ with ResNet-50 backbone, MSCOCO pretrained) Validation mIoU 64.9% # 1
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 64.4% # 11
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 65.9% # 11
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 71.6% # 13
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 67.9% # 2
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled SemiSegContrast (DeepLab v2 with ResNet-101 backbone, MSCOCO pretrained) Validation mIoU 70.0% # 4

Methods