An Incremental Learning Approach for Sustainable Region Isolation and Integration

Humans are capable of constantly acquiring new knowledge, integrating and optimizing old knowledge without forgetting it. In this paper, we mimic partitioned learning and memory replay in the human brain and propose an incremental learning approach named SRII, short for sustainable regional isolation and integration. SRII consists of two phases, "regional isolation" and "regional integration", which are iterated alternately to achieve continuous class incremental learning. "Regional isolation" isolates new learning processes to avoid interfering with existing knowledge, while "regional integration" establishes a unified, high-precision cognition to accommodate the single-headed output requirements of class incremental learning. Moreover, SRII employs a dual branch fusion to boost the network's adaptability to new knowledge, and a margin loss regularization item to address "recency bias". Comprehensive ablation studies are also undertaken to evaluate the positive effects of each component of SRII. Experimental results on the CIFAR100 and miniImageNet dataset demonstrate that SRII outperforms a number of state-of-the-art technologies. In all 5-stage and 10-stage incremental settings, our approach consistently outperforms the baseline and achieves at least 5.27%+ average accuracy improvement.