MetaFormer: A Unified Meta Framework for Fine-Grained Recognition

5 Mar 2022  ·  Qishuai Diao, Yi Jiang, Bin Wen, Jia Sun, Zehuan Yuan ·

Fine-Grained Visual Classification(FGVC) is the task that requires recognizing the objects belonging to multiple subordinate categories of a super-category. Recent state-of-the-art methods usually design sophisticated learning pipelines to tackle this task. However, visual information alone is often not sufficient to accurately differentiate between fine-grained visual categories. Nowadays, the meta-information (e.g., spatio-temporal prior, attribute, and text description) usually appears along with the images. This inspires us to ask the question: Is it possible to use a unified and simple framework to utilize various meta-information to assist in fine-grained identification? To answer this problem, we explore a unified and strong meta-framework(MetaFormer) for fine-grained visual classification. In practice, MetaFormer provides a simple yet effective approach to address the joint learning of vision and various meta-information. Moreover, MetaFormer also provides a strong baseline for FGVC without bells and whistles. Extensive experiments demonstrate that MetaFormer can effectively use various meta-information to improve the performance of fine-grained recognition. In a fair comparison, MetaFormer can outperform the current SotA approaches with only vision information on the iNaturalist2017 and iNaturalist2018 datasets. Adding meta-information, MetaFormer can exceed the current SotA approaches by 5.9% and 5.3%, respectively. Moreover, MetaFormer can achieve 92.3% and 92.7% on CUB-200-2011 and NABirds, which significantly outperforms the SotA approaches. The source code and pre-trained models are released at

PDF Abstract

Results from the Paper

 Ranked #1 on Fine-Grained Image Classification on NABirds (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Fine-Grained Image Classification CUB-200-2011 MetaFormer (MetaFormer-2,384) Accuracy 92.9% # 2
Image Classification iNaturalist MetaFormer (MetaFormer-2,384) Top 1 Accuracy 80.4% # 4
Image Classification iNaturalist MetaFormer (MetaFormer-2,384,extra_info) Top 1 Accuracy 83.4% # 2
Image Classification iNaturalist 2018 MetaFormer (MetaFormer-2,384,extra_info) Top-1 Accuracy 88.7% # 5
Image Classification iNaturalist 2018 MetaFormer (MetaFormer-2,384) Top-1 Accuracy 84.3% # 10
Fine-Grained Image Classification NABirds MetaFormer (MetaFormer-2,384) Accuracy 93.0% # 1