FOSTER: Feature Boosting and Compression for Class-Incremental Learning

10 Apr 2022  ·  Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, De-Chuan Zhan ·

The ability to learn new concepts continually is necessary in this ever-changing world. However, deep neural networks suffer from catastrophic forgetting when learning new categories. Many works have been proposed to alleviate this phenomenon, whereas most of them either fall into the stability-plasticity dilemma or take too much computation or storage overhead. Inspired by the gradient boosting algorithm to gradually fit the residuals between the target model and the previous ensemble model, we propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively. Specifically, we first dynamically expand new modules to fit the residuals between the target and the output of the original model. Next, we remove redundant parameters and feature dimensions through an effective distillation strategy to maintain the single backbone model. We validate our method FOSTER on CIFAR-100 and ImageNet-100/1000 under different settings. Experimental results show that our method achieves state-of-the-art performance. Code is available at: https://github.com/G-U-N/ECCV22-FOSTER.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Incremental Learning CIFAR-100 - 50 classes + 10 steps of 5 classes FOSTER Average Incremental Accuracy 67.95 # 2
Incremental Learning CIFAR-100 - 50 classes + 25 steps of 2 classes FOSTER Average Incremental Accuracy 63.83 # 2
Incremental Learning CIFAR-100 - 50 classes + 5 steps of 10 classes FOSTER Average Incremental Accuracy 69.46 # 2
Incremental Learning CIFAR100-B0(10steps of 10 classes) FOSTER Average Incremental Accuracy 72.9 # 2
Incremental Learning CIFAR100B020Step(5ClassesPerStep) FOSTER Average Incremental Accuracy 70.65 # 2
Incremental Learning ImageNet100 - 10 steps FOSTER Average Incremental Accuracy 77.75 # 2
Incremental Learning ImageNet100 - 20 steps FOSTER Average Incremental Accuracy 74.49 # 1
Incremental Learning ImageNet-100 - 50 classes + 10 steps of 5 classes FOSTER Average Incremental Accuracy 77.54 # 3
Incremental Learning ImageNet-100 - 50 classes + 25 steps of 2 classes FOSTER Average Incremental Accuracy 69.34 # 2
Incremental Learning ImageNet-100 - 50 classes + 5 steps of 10 classes FOSTER Average Incremental Accuracy 80.22 # 1
Incremental Learning ImageNet - 10 steps FOSTER Average Incremental Accuracy 68.34 # 3

Methods