VOLO: Vision Outlooker for Visual Recognition

24 Jun 2021  ·  Li Yuan, Qibin Hou, Zihang Jiang, Jiashi Feng, Shuicheng Yan ·

Visual recognition has been dominated by convolutional neural networks (CNNs) for years. Though recently the prevailing vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification, their performance is still inferior to that of the latest SOTA CNNs if no extra data are provided. In this work, we try to close the performance gap and demonstrate that attention-based models are indeed able to outperform CNNs. We find a major factor limiting the performance of ViTs for ImageNet classification is their low efficacy in encoding fine-level features into the token representations. To resolve this, we introduce a novel outlook attention and present a simple and general architecture, termed Vision Outlooker (VOLO). Unlike self-attention that focuses on global dependency modeling at a coarse level, the outlook attention efficiently encodes finer-level features and contexts into tokens, which is shown to be critically beneficial to recognition performance but largely ignored by the self-attention. Experiments show that our VOLO achieves 87.1% top-1 accuracy on ImageNet-1K classification, which is the first model exceeding 87% accuracy on this competitive benchmark, without using any extra training data In addition, the pre-trained VOLO transfers well to downstream tasks, such as semantic segmentation. We achieve 84.3% mIoU score on the cityscapes validation set and 54.3% on the ADE20K validation set. Code is available at \url{https://github.com/sail-sg/volo}.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semantic Segmentation ADE20K VOLO-D5 Validation mIoU 54.3 # 56
Semantic Segmentation Cityscapes val VOLO-D4 (MS, ImageNet1k pretrain) mIoU 84.3 # 16
Semantic Segmentation Graz-02 VOLO-D5 Pixel Accuracy 85 # 1
Image Classification ImageNet VOLO-D1 Top 1 Accuracy 85.2% # 239
Number of params 27M # 614
Image Classification ImageNet VOLO-D4 Top 1 Accuracy 86.8% # 120
Number of params 193M # 896
Image Classification ImageNet VOLO-D5 Top 1 Accuracy 87.1% # 102
Number of params 296M # 915
Hardware Burden None # 1
Operations per network pass None # 1
Image Classification ImageNet VOLO-D3 Top 1 Accuracy 86.3% # 152
Number of params 86M # 815
Image Classification ImageNet VOLO-D2 Top 1 Accuracy 86% # 175
Number of params 59M # 762
Image Classification ImageNet ReaL VOLO-D4 Accuracy 90.5% # 18
Image Classification ImageNet ReaL VOLO-D5 Accuracy 90.6% # 14
Image Classification ImageNet V2 VOLO-D5 Top 1 Accuracy 78 # 14
Image Classification ImageNet V2 VOLO-D4 Top 1 Accuracy 77.8 # 15
Domain Generalization VizWiz-Classification VOLO-D5 Accuracy - All Images 57.2 # 1
Accuracy - Corrupted Images 51.8 # 1
Accuracy - Clean Images 59.7 # 1
Image Classification VizWiz-Classification VOLO-D5 Accuracy 57.2 # 1


No methods listed for this paper. Add relevant methods here