RMM: Reinforced Memory Management for Class-Incremental Learning

NeurIPS 2021  ยท  Yaoyao Liu, Bernt Schiele, Qianru Sun ยท

Class-Incremental Learning (CIL) [40] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the past, and future data are strictly non-accessible during the incremental phases. We solve this by training the policy function of RMM on pseudo CIL tasks, e.g., the tasks built on the data of the 0-th phase, and then applying it to target tasks. RMM propagates two levels of actions: Level-1 determines how to split the memory between old and new classes, and Level-2 allocates memory for each specific class. In essence, it is an optimizable and general method for memory management that can be used in any replaying-based CIL method. For evaluation, we plug RMM into two top-performing baselines (LUCIR+AANets and POD+AANets [30]) and conduct experiments on three benchmarks (CIFAR-100, ImageNet-Subset, and ImageNet-Full). Our results show clear improvements, e.g., boosting POD+AANets by 3.6%, 4.4%, and 1.9% in the 25-Phase settings of the above benchmarks, respectively.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Incremental Learning CIFAR-100 - 50 classes + 10 steps of 5 classes RMM (Modified ResNet-32) Average Incremental Accuracy 67.61 # 3
Incremental Learning CIFAR-100 - 50 classes + 25 steps of 2 classes RMM (Modified ResNet-32) Average Incremental Accuracy 66.21 # 1
Incremental Learning CIFAR-100 - 50 classes + 5 steps of 10 classes RMM (Modified ResNet-32) Average Incremental Accuracy 68.86 # 3
Incremental Learning ImageNet100 - 10 steps RMM (ResNet-18) Average Incremental Accuracy 78.47 # 1
Incremental Learning ImageNet-100 - 50 classes + 10 steps of 5 classes RMM (ResNet-18) Average Incremental Accuracy 78.47 # 1
Incremental Learning ImageNet-100 - 50 classes + 25 steps of 2 classes RMM (ResNet-18) Average Incremental Accuracy 76.54 # 1
Incremental Learning ImageNet-100 - 50 classes + 5 steps of 10 classes RMM (ResNet-18) Average Incremental Accuracy 79.52 # 2
Incremental Learning ImageNet - 10 steps RMM (ResNet-18) Average Incremental Accuracy 67.45 # 4
Incremental Learning ImageNet - 500 classes + 10 steps of 50 classes RMM (ResNet-18) Average Incremental Accuracy 67.45 # 1
Incremental Learning ImageNet - 500 classes + 25 steps of 20 classes RMM (ResNet-18) Average Incremental Accuracy 63.93 # 1
Incremental Learning ImageNet - 500 classes + 5 steps of 100 classes RMM (ResNet-18) Average Incremental Accuracy 69.21 # 1

Methods