Boosting Co-teaching with Compression Regularization for Label Noise

28 Apr 2021  ·  Yingyi Chen, Xi Shen, Shell Xu Hu, Johan A. K. Suykens ·

In this paper, we study the problem of learning image classification models in the presence of label noise. We revisit a simple compression regularization named Nested Dropout. We find that Nested Dropout, though originally proposed to perform fast information retrieval and adaptive data compression, can properly regularize a neural network to combat label noise. Moreover, owing to its simplicity, it can be easily combined with Co-teaching to further boost the performance. Our final model remains simple yet effective: it achieves comparable or even better performance than the state-of-the-art approaches on two real-world datasets with label noise which are Clothing1M and ANIMAL-10N. On Clothing1M, our approach obtains 74.9% accuracy which is slightly better than that of DivideMix. On ANIMAL-10N, we achieve 84.1% accuracy while the best public result by PLC is 83.4%. We hope that our simple approach can be served as a strong baseline for learning with label noise. Our implementation is available at

PDF Abstract

Results from the Paper

Ranked #12 on Image Classification on Clothing1M (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Learning with noisy labels ANIMAL Nested Dropout Accuracy 81.3 # 15
Network Vgg19-BN # 1
ImageNet Pretrained NO # 1
Learning with noisy labels ANIMAL CE + Dropout Accuracy 81.3 # 15
Network Vgg19-BN # 1
ImageNet Pretrained NO # 1
Image Classification Clothing1M NestedCoTeaching Accuracy 74.9% # 12