Co-variance: Tackling Noisy Labels with Sample Selection by Emphasizing High-variance Examples

The sample selection approach is popular in learning with noisy labels, which tends to select potentially clean data out of noisy data for robust training. The state-of-the-art methods train two deep networks simultaneously for sample selection, which aims to employ their different learning abilities. To prevent two networks from converging to a consensus, their divergence should be maintained during training. Typically, the divergence is kept by first locating the disagreement data on which the prediction labels of two networks are different, and then selecting clean data out of such data. However, this procedure is sample-inefficient for network weight updates, which means that a few clean examples can be utilized in training. In this paper, to address the issues, we propose a simple yet effective method called Co-variance. In particular, we select possibly clean data that simultaneously have high-variance prediction probabilities between two networks. As selected data have high variances, the divergence of two networks can be maintained by training on such data. Additionally, the condition of high variances is milder than the condition of disagreement in sample selection, which allows more data to be considered for training, and makes our method more sample-efficient. Moreover, we show that the proposed method enables to mine enough hard clean examples to help generalization. A series of empirical results show that Co-variance is superior to multiple baselines in the robustness of trained models, especially on class-imbalanced and real-world noisy datasets.