Improving robustness of softmax corss-entropy loss via inference information
Adversarial examples easily mislead the vision systems based on deep neural networks (DNNs) which are trained with the softmax cross entropy (SCE) loss. Such a vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training samples, whereas the resultant feature distributions between the training and adversarial samples are unfortunately misaligned. Several state-of-the-arts start from improving the inter-class separability of training samples by modifying loss functions, where we argue that the adversarial samples are ignored and thus limited robustness to adversarial attacks is resulted. In this paper, we exploit the distribution difference between the ground-truth and the predictions measured by the Kullback–Leibler divergence. The difference is termed as inference region and inspires us to involve an additive inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information is a guarantee to both the inter-class separability and the improved generalization to adversarial samples, which is furthermore demonstrated under the min-max framework. Extensive experiments show that under strong adaptive attacks, the DNN models trained with the proposed I-SCE loss achieve superior performance and robustness over the state-of-the-arts.
PDF Abstract
CIFAR-10
MNIST
