Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation

13 Oct 2022  ·  Dipam Goswami, René Schuster, Joost Van de Weijer, Didier Stricker ·

In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. Although recent works focused on these issues, existing classifier initialization methods do not address the background shift problem and assign the same initialization weights to both background and new foreground class classifiers. We propose to address the background shift with a novel classifier initialization method which employs gradient-based attribution to identify the most relevant weights for new classes from the classifier's weights for the previous background and transfers these weights to the new classifier. This warm-start weight initialization provides a general solution applicable to several CISS methods. Furthermore, it accelerates learning of new classes while mitigating forgetting. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Overlapped 50-50 ADE20K MiB+AWT mIoU 33.5 # 1
Overlapped 100-10 ADE20K MiB+AWT Mean IoU (test) 33.2 # 2
Overlapped 100-50 ADE20K MiB+AWT mIoU 35.6 # 1
Overlapped 100-5 ADE20K MiB+AWT mIoU 31.1 # 3
Overlapped 14-1 Cityscapes MiB+AWT mIoU 46.9 # 1
Overlapped 10-1 Cityscapes MiB+AWT mIoU 44.9 # 1
Overlapped 15-1 PASCAL VOC 2012 SSUL+AWT mIoU 67.6 # 5
Overlapped 15-5 PASCAL VOC 2012 SSUL+AWT Mean IoU (val) 71.4 # 5
Overlapped 10-1 PASCAL VOC 2012 SSUL+AWT mIoU 60.7 # 4
Overlapped 5-3 PASCAL VOC 2012 SSUL+AWT Mean IoU (test) 57.1 # 3


No methods listed for this paper. Add relevant methods here