Modeling Unknown Semantic Labels as Uncertainty in the Prediction: Evidential Deep Learning for Class-Incremental Semantic Segmentation

Class-Incremental Learning is an essential component for expanding the knowledge of previously trained neural networks. This is especially useful if the system needs to be able to handle new objects but the original training data is unavailable. While the semantic segmentation problem has received less attention than classification, it is faced with its own set of challenges in terms of unlabeled classes in the images. In this paper we address the problem of how to model unlabeled classes to avoid unnecessary feature clustering of uncorrelated classes. We propose to use Evidential Deep Learning to model the evidence of the classes with a Dirichlet distribution. Our method factorizes the problem into a separate foreground class probability, calculated by the expected value of the Dirichlet distribution, and an unknown class probability corresponding to the uncertainty of the estimate. In our novel formulation the background probability is implicitly modelled, avoiding the feature space clustering that comes from forcing the model to output a high background score for these pixels. Experiments on the incremental Pascal VOC and ADE20k show that our method is superior to state-of-the-art methods, especially when repeatedly learning new classes.