Grafit: Learning fine-grained image representations with coarse labels

This paper tackles the problem of learning a finer representation than the one provided by training labels. This enables fine-grained category retrieval of images in a collection annotated with coarse labels only. Our network is learned with a nearest-neighbor classifier objective, and an instance loss inspired by self-supervised learning. By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods. Our strategy outperforms all competing methods for retrieving or classifying images at a finer granularity than that available at train time. It also improves the accuracy for transfer learning tasks to fine-grained datasets, thereby establishing the new state of the art on five public benchmarks, like iNaturalist-2018.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification CIFAR-100 Grafit (ResNet-50) Percentage correct 83.7 # 73
Image Classification Flowers-102 Grafit (RegNet-8GF) Accuracy 99.1% # 12
Fine-Grained Image Classification Food-101 Grafit (RegNet-8GF) Accuracy 93.7 # 3
Image Classification ImageNet Grafit (ResNet-50) Top 1 Accuracy 79.6% # 472
Hardware Burden None # 1
Operations per network pass None # 1
Image Classification iNaturalist 2018 RegNet-8GF Top-1 Accuracy 81.2% # 7
Image Classification iNaturalist 2019 Grafit (RegnetY 8GF) Top-1 Accuracy 84.1 # 2
Fine-Grained Image Classification Oxford 102 Flowers Grafit (RegNet-8GF) Accuracy 99.1% # 4
Fine-Grained Image Classification Stanford Cars Grafit (RegNet-8GF) Accuracy 94.7% # 22

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Uses Extra
Training Data
Source Paper Compare
Image Classification iNaturalist 2018 ResNet-50 Top-1 Accuracy 69.8% # 28

Methods


No methods listed for this paper. Add relevant methods here