Billion-scale semi-supervised learning for image classification

2 May 2019  ·  I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, Dhruv Mahajan ·

This paper presents a study of semi-supervised learning with large convolutional networks. We propose a pipeline, based on a teacher/student paradigm, that leverages a large collection of unlabelled images (up to 1 billion)... Our main goal is to improve the performance for a given target architecture, like ResNet-50 or ResNext. We provide an extensive analysis of the success factors of our approach, which leads us to formulate some recommendations to produce high-accuracy models for image classification with semi-supervised learning. As a result, our approach brings important gains to standard architectures for image, video and fine-grained classification. For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark. read more

PDF Abstract


Introduced in the Paper:


Used in the Paper:

ImageNet CUB-200-2011 YFCC100M

Results from the Paper

Ranked #85 on Image Classification on ImageNet (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification ImageNet ResNeXt-101 32x8d (semi-weakly sup.) Top 1 Accuracy 84.3% # 95
Top 5 Accuracy 97.2% # 29
Number of params 88M # 59
Image Classification ImageNet ResNet-50 (semi-weakly sup.) Top 1 Accuracy 81.2% # 234
Image Classification ImageNet ResNeXt-101 32x4d (semi-weakly sup.) Top 1 Accuracy 83.4% # 131
Top 5 Accuracy 96.8% # 42
Number of params 42M # 148
Image Classification ImageNet ResNeXt-101 32x16d (semi-weakly sup.) Top 1 Accuracy 84.8% # 85
Top 5 Accuracy 97.4% # 24
Number of params 193M # 39