Bootstrapping Semantic Segmentation with Regional Contrast

ICLR 2022  ·  Shikun Liu, Shuaifeng Zhi, Edward Johns, Andrew J. Davison ·

We present ReCo, a contrastive learning framework designed at a regional level to assist learning in semantic segmentation. ReCo performs semi-supervised or supervised pixel-level contrastive learning on a sparse set of hard negative pixels, with minimal additional memory footprint. ReCo is easy to implement, being built on top of off-the-shelf segmentation networks, and consistently improves performance in both semi-supervised and supervised semantic segmentation methods, achieving smoother segmentation boundaries and faster convergence. The strongest effect is in semi-supervised learning with very few labels. With ReCo, we achieve high-quality semantic segmentation models, requiring only 5 examples of each semantic class. Code is available at https://github.com/lorenmt/reco.

PDF Abstract ICLR 2022 PDF ICLR 2022 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data 		Result Benchmark
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 56.53% # 9
Compare
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 60.28% # 5
Compare
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 64.94% # 20
Compare
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 66.44% # 18
Compare
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 67.53% # 17
Compare
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 68.50% # 16
Compare
Semi-Supervised Semantic Segmentation Cityscapes 50% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 68.69% # 16
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 12.5% labeled ReCo Validation mIoU 71.00% # 23
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 1% labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pre-trained) Validation mIoU 63.60% # 1
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 1% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pre-trained) Validation mIoU 63.16% # 2
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 72.14% # 1
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 2% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 66.41% # 6
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled ReCo (DeepLab v2 with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 68.85% # 8
Compare
Semi-Supervised Semantic Segmentation Pascal VOC 2012 5% labeled ReCo (DeepLab v3+ with ResNet-101 backbone, ImageNet pretrained) Validation mIoU 73.66% # 1
Compare

Methods
Add Remove

Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy