Representation Compensation Networks for Continual Semantic Segmentation
In this work, we study the continual semantic segmentation problem, where the deep neural networks are required to incorporate new classes continually without catastrophic forgetting. We propose to use a structural re-parameterization mechanism, named representation compensation (RC) module, to decouple the representation learning of both old and new knowledge. The RC module consists of two dynamically evolved branches with one frozen and one trainable. Besides, we design a pooled cube knowledge distillation strategy on both spatial and channel dimensions to further enhance the plasticity and stability of the model. We conduct experiments on two challenging continual semantic segmentation scenarios, continual class segmentation and continual domain segmentation. Without any extra computational overhead and parameters during inference, our method outperforms state-of-the-art performance. The code is available at \url{https://github.com/zhangchbin/RCIL}.
PDF Abstract CVPR 2022 PDF CVPR 2022 AbstractCode
Tasks
Class Incremental Learning
Continual Learning
Continual Semantic Segmentation
Disjoint 10-1
Disjoint 15-1
Disjoint 15-5
Domain 1-1
Domain 11-1
Domain 11-5
Knowledge Distillation
Overlapped 100-10
Overlapped 100-5
Overlapped 100-50
Overlapped 10-1
Overlapped 15-1
Overlapped 15-5
Overlapped 50-50
Segmentation
Semantic Segmentation
ImageNet
Cityscapes
ADE20K
