Prior Guided Feature Enrichment Network for Few-Shot Segmentation

4 Aug 2020  ·  Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang, Ruiyu Li, Jiaya Jia ·

State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results and hardly work on unseen classes without fine-tuning. Few-shot segmentation is thus proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information of training classes and spatial inconsistency between query and support targets. To alleviate these issues, we propose the Prior Guided Feature Enrichment Network (PFENet). It consists of novel designs of (1) a training-free prior mask generation method that not only retains generalization power but also improves model performance and (2) Feature Enrichment Module (FEM) that overcomes spatial inconsistency by adaptively enriching query features with support features and prior masks. Extensive experiments on PASCAL-5$^i$ and COCO prove that the proposed prior generation method and FEM both improve the baseline method significantly. Our PFENet also outperforms state-of-the-art methods by a large margin without efficiency loss. It is surprising that our model even generalizes to cases without labeled support samples. Our code is available at https://github.com/Jia-Research-Lab/PFENet/.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Semantic Segmentation COCO-20i (10-shot) PFENet Mean IoU 39.7 # 3
Few-Shot Semantic Segmentation COCO-20i -> Pascal VOC (1-shot) PFENet Mean IoU 61.1 # 2
Few-Shot Semantic Segmentation COCO-20i -> Pascal VOC (5-shot) PFENet Mean IoU 63.4 # 2
Few-Shot Semantic Segmentation PASCAL-5i (10-Shot) PFENet Mean IoU 62.1 # 3
Few-Shot Semantic Segmentation PASCAL-5i (1-Shot) PFENet Mean IoU 60.8 # 4
Few-Shot Semantic Segmentation PASCAL-5i (5-Shot) PFENet Mean IoU 61.9 # 6

Methods


No methods listed for this paper. Add relevant methods here