Puzzle-CAM: Improved localization via matching partial and full features

27 Jan 2021  ·  Sanghyun Jo, In-Jae Yu ·

Weakly-supervised semantic segmentation (WSSS) is introduced to narrow the gap for semantic segmentation performance from pixel-level supervision to image-level supervision. Most advanced approaches are based on class activation maps (CAMs) to generate pseudo-labels to train the segmentation network... The main limitation of WSSS is that the process of generating pseudo-labels from CAMs that use an image classifier is mainly focused on the most discriminative parts of the objects. To address this issue, we propose Puzzle-CAM, a process that minimizes differences between the features from separate patches and the whole image. Our method consists of a puzzle module and two regularization terms to discover the most integrated region in an object. Puzzle-CAM can activate the overall region of an object using image-level supervision without requiring extra parameters. % In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 test dataset. In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 dataset. Code associated with our experiments is available at https://github.com/OFRIN/PuzzleCAM. read more

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 test Puzzle-CAM (ResNeSt-101) Mean IoU 67.7 # 11
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 test Puzzle-CAM (ResNeSt-269) Mean IoU 72.2 # 1
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 val Puzzle-CAM (ResNeSt-269) Mean IoU 71.9 # 1
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 val Puzzle-CAM (ResNeSt-101) Mean IoU 66.9 # 14

Methods


No methods listed for this paper. Add relevant methods here