PLOP: Learning without Forgetting for Continual Semantic Segmentation
Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks.
PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract
Tasks
Class Incremental Learning
Continual Learning
Continual Semantic Segmentation
Disjoint 10-1
Disjoint 15-1
Disjoint 15-5
Domain 1-1
Domain 11-1
Domain 11-5
Overlapped 100-10
Overlapped 100-5
Overlapped 100-50
Overlapped 10-1
Overlapped 15-1
Overlapped 15-5
Overlapped 19-1
Overlapped 50-50
Segmentation
Semantic Segmentation
ImageNet
Cityscapes
ADE20K
PASCAL VOC
