PLOP: Learning without Forgetting for Continual Semantic Segmentation

Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes... However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks. read more

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Overlapped 50-50 ADE20K MiB Mean IoU 29.31 # 2
Overlapped 100-5 ADE20K PLOP Mean IoU 28.75 # 1
Overlapped 100-10 ADE20K MiB Mean IoU 29.24 # 2
Overlapped 100-50 ADE20K PLOP Mean IoU 32.94 # 1
Overlapped 50-50 ADE20K PLOP Mean IoU 30.4 # 1
Overlapped 100-10 ADE20K PLOP Mean IoU 31.59 # 1
Overlapped 100-50 ADE20K MiB Mean IoU 32.79 # 2
Overlapped 100-5 ADE20K MiB Mean IoU 25.96 # 2
Domain 11-5 Cityscapes val MiB Mean IoU 61.51 # 2
Domain 1-1 Cityscapes val MiB Mean IoU 42.15 # 2
Domain 11-5 Cityscapes val PLOP Mean IoU 63.51 # 1
Domain 11-1 Cityscapes val PLOP Mean IoU 62.05 # 1
Domain 1-1 Cityscapes val PLOP Mean IoU 45.24 # 1
Domain 11-1 Cityscapes val MiB Mean IoU 60.02 # 2
Overlapped 19-1 PASCAL VOC 2012 PLOP Mean IoU (val) 73.54 # 1
Overlapped 15-1 PASCAL VOC 2012 MiB Mean IoU (val) 29.29 # 2
Overlapped 15-5 PASCAL VOC 2012 PLOP Mean IoU (val) 70.09 # 1
Overlapped 15-1 PASCAL VOC 2012 PLOP Mean IoU (val) 54.64 # 1
Overlapped 19-1 PASCAL VOC 2012 MiB Mean IoU (val) 69.15 # 2
Overlapped 10-1 PASCAL VOC 2012 MiB Mean IoU (val) 12.65 # 2
Overlapped 10-1 PASCAL VOC 2012 PLOP Mean IoU (val) 30.45 # 1
Overlapped 15-5 PASCAL VOC 2012 MiB Mean IoU (val) 70.08 # 2

Methods


No methods listed for this paper. Add relevant methods here