Revisiting Weakly Supervised Pre-Training of Visual Perception Models

Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent studies suggest that large-scale weakly supervised pre-training can outperform fully supervised approaches. This paper revisits weakly-supervised pre-training of models using hashtag supervision with modern versions of residual networks and the largest-ever dataset of images and corresponding hashtags. We study the performance of the resulting models in various transfer-learning settings including zero-shot transfer. We also compare our models with those obtained via large-scale self-supervised learning. We find our weakly-supervised models to be very competitive across all settings, and find they substantially outperform their self-supervised counterparts. We also include an investigation into whether our models learned potentially troubling associations or stereotypes. Overall, our results provide a compelling argument for the use of weakly supervised learning in the development of visual recognition systems. Our models, Supervised Weakly through hashtAGs (SWAG), are available publicly.

PDF Abstract

Results from the Paper

 Ranked #1 on Image Classification on Places365-Standard (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Fine-Grained Image Classification CUB-200-2011 SWAG (ViT H/14) Accuracy 91.7 # 3
Image Classification ImageNet SWAG (ViT H/14) Top 1 Accuracy 88.6% # 21
Image Classification ImageNet ReaL SWAG (RegNetY 128GF) Accuracy 90.7% # 10
Image Classification ImageNet V2 SWAG (ViT H/14) Top 1 Accuracy 81.1 # 4
Image Classification iNaturalist 2018 SWAG (ViT H/14) Top-1 Accuracy 86.0% # 3
Image Classification ObjectNet SWAG (ViT H/14) Top-1 Accuracy 69.5 # 4
Image Classification Places365-Standard SWAG (ViT H/14) Top 1 Accuracy 60.7 # 1


No methods listed for this paper. Add relevant methods here