Gradient Imbalance and solution in Online Continual learning

Most existing techniques for online continual learning are based on experience-replay. In this approach, a memory buffer is used to save some data from past tasks for dealing with catastrophic forgetting. In training, a small batch of data from the data stream of the current task and some sampled data from a memory buffer are used jointly to update or train the current model. In this paper, we study the experience replay-based approach from a new angle, gradient imbalance. We first investigate and analyze this phenomenon experimentally from two perspectives: imbalance of samples introduced by experience replay and sequence of classes introduced by incremental learning. To our knowledge, this problem has not been studied before and it significantly limits the performance of online continual learning. Based on observations from experiments and theoretical analysis, a new learning strategy and a new loss are proposed to deal with the problem. Empirical evaluation shows that GAD helps improve the online CL performance by more than 11% in accuracy.