Generalized Few-shot Semantic Segmentation

Training semantic segmentation models requires a large amount of finely annotated data, making it hard to quickly adapt to novel classes not satisfying this condition. Few-Shot Segmentation (FS-Seg) tackles this problem with many constraints. In this paper, we introduce a new benchmark, called Generalized Few-Shot Semantic Segmentation (GFS-Seg), to analyze the generalization ability of simultaneously segmenting the novel categories with very few examples and the base categories with sufficient examples. It is the first study showing that previous representative state-of-the-art FS-Seg methods fall short in GFS-Seg and the performance discrepancy mainly comes from the constrained setting of FS-Seg. To make GFS-Seg tractable, we set up a GFS-Seg baseline that achieves decent performance without structural change on the original model. Then, since context is essential for semantic segmentation, we propose the Context-Aware Prototype Learning (CAPL) that significantly improves performance by 1) leveraging the co-occurrence prior knowledge from support samples, and 2) dynamically enriching contextual information to the classifier, conditioned on the content of each query image. Both two contributions are experimentally shown to have substantial practical merit. Extensive experiments on Pascal-VOC and COCO manifest the effectiveness of CAPL, and CAPL generalizes well to FS-Seg by achieving competitive performance. Code is available at https://github.com/dvlab-research/GFS-Seg.

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Generalized Few-Shot Semantic Segmentation COCO-20i (1-shot) CAPL (ResNet-50) Mean IoU 35.46 # 3
Mean Base and Novel 25.83 # 3
Generalized Few-Shot Semantic Segmentation COCO-20i (5-shot) CAPL (ResNet-50) Mean IoU 36.8 # 3
Mean Base and Novel 28.15 # 3
Generalized Few-Shot Semantic Segmentation PASCAL-5i (1-Shot) CAPL (ResNet-50) Mean IoU 54.38 # 3
Mean Base and Novel 42.17 # 3
Generalized Few-Shot Semantic Segmentation PASCAL-5i (5-Shot) CAPL (ResNet-50) Mean IoU 55.72 # 4
Mean Base and Novel 44.28 # 4

Methods


No methods listed for this paper. Add relevant methods here