DER: Dynamically Expandable Representation for Class Incremental Learning

CVPR 2021  ·  Shipeng Yan, Jiangwei Xie, Xuming He ·

We address the problem of class incremental learning, which is a core step towards achieving adaptive vision intelligence. In particular, we consider the task setting of incremental learning with limited memory and aim to achieve better stability-plasticity trade-off. To this end, we propose a novel two-stage learning approach that utilizes a dynamically expandable representation for more effective incremental concept modeling. Specifically, at each incremental step, we freeze the previously learned representation and augment it with additional feature dimensions from a new learnable feature extractor. This enables us to integrate new visual concepts with retaining learned knowledge. We dynamically expand the representation according to the complexity of novel concepts by introducing a channel-level mask-based pruning strategy. Moreover, we introduce an auxiliary loss to encourage the model to learn diverse and discriminate features for novel concepts. We conduct extensive experiments on the three class incremental learning benchmarks and our method consistently outperforms other methods with a large margin.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Incremental Learning CIFAR-100 - 50 classes + 10 steps of 5 classes DER(Modified ResNet-32) Average Incremental Accuracy 66.36 # 4
Incremental Learning CIFAR-100 - 50 classes + 10 steps of 5 classes DER(Standard ResNet-18) Average Incremental Accuracy 72.45 # 1
Incremental Learning CIFAR-100 - 50 classes + 5 steps of 10 classes DER(Standard ResNet-18) Average Incremental Accuracy 72.60 # 1
Incremental Learning CIFAR-100 - 50 classes + 5 steps of 10 classes DER(Modified Res-32) Average Incremental Accuracy 67.60 # 4
Incremental Learning CIFAR100-B0(10steps of 10 classes) DER(ResNet-18) Average Incremental Accuracy 74.64 # 1
Incremental Learning CIFAR100B020Step(5ClassesPerStep) DER(ResNet-18) Average Incremental Accuracy 73.98 # 1
Incremental Learning CIFAR100B050S(2ClassesPerStep) DER(ResNet-18) Average Incremental Accuracy 72.05 # 1
Incremental Learning ImageNet100 - 10 steps DER w/o Pruning Average Incremental Accuracy 77.18 # 3
Final Accuracy 66.70 # 2
Average Incremental Accuracy Top-5 93.23 # 1
Final Accuracy Top-5 87.52 # 3
# M Params 112.27 # 6
Incremental Learning ImageNet100 - 10 steps DER Average Incremental Accuracy 76.12 # 5
Final Accuracy 66.07 # 3
Average Incremental Accuracy Top-5 92.79 # 2
Final Accuracy Top-5 88.38 # 1
Incremental Learning ImageNet-100 - 50 classes + 10 steps of 5 classes DER Average Incremental Accuracy 77.73 # 2
Incremental Learning ImageNet - 10 steps DER Average Incremental Accuracy 66.73 # 5
Final Accuracy 58.62 # 3
Average Incremental Accuracy Top-5 87.08 # 3
Final Accuracy Top-5 81.89 # 3
Incremental Learning ImageNet - 10 steps DER w/o Pruning Average Incremental Accuracy 68.84 # 2
Final Accuracy 60.16 # 2
Average Incremental Accuracy Top-5 88.17 # 2
Final Accuracy Top-5 82.86 # 2
# M Params 116.89 # 6

Methods


No methods listed for this paper. Add relevant methods here