ResMLP: Feedforward networks for image classification with data-efficient training

We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification. It is a simple residual network that alternates (i) a linear layer in which image patches interact, independently and identically across channels, and (ii) a two-layer feed-forward network in which channels interact independently per patch. When trained with a modern training strategy using heavy data-augmentation and optionally distillation, it attains surprisingly good accuracy/complexity trade-offs on ImageNet. We also train ResMLP models in a self-supervised setup, to further remove priors from employing a labelled dataset. Finally, by adapting our model to machine translation we achieve surprisingly good results. We share pre-trained models and our code based on the Timm library.

PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 Abstract

Results from the Paper


Ranked #5 on Image Classification on ImageNet ReaL (Top 1 Accuracy metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification CIFAR-10 ResMLP-24 Percentage correct 98.7 # 22
Top-1 Accuracy 98.7 # 6
Image Classification CIFAR-10 ResMLP-12 Percentage correct 98.1 # 45
Top-1 Accuracy 98.1 # 10
Image Classification CIFAR-100 ResMLP-12 Percentage correct 87.0 # 46
Image Classification CIFAR-100 ResMLP-24 Percentage correct 89.5 # 28
Image Classification Flowers-102 ResMLP24 Accuracy 97.9 # 28
Image Classification Flowers-102 ResMLP12 Accuracy 97.4 # 35
Image Classification ImageNet ResMLP-12 (distilled, class-MLP) Top 1 Accuracy 78.6% # 509
Number of params 17.7M # 342
GFLOPs 3 # 143
Image Classification ImageNet ResMLP-B24/8 Top 1 Accuracy 83.6% # 222
Number of params 116M # 607
Image Classification ImageNet ResMLP-S24 Top 1 Accuracy 80.8% # 406
Number of params 30M # 434
GFLOPs 6 # 196
Self-Supervised Image Classification ImageNet DINO (ResMLP-12) Top 1 Accuracy 67.5% # 79
Number of Params 15M # 65
Top 1 Accuracy (kNN, k=20) 62.6% # 19
Self-Supervised Image Classification ImageNet DINO (ResMLP-24) Top 1 Accuracy 72.8% # 66
Number of Params 30M # 40
Top 1 Accuracy (kNN, k=20) 69.4% # 17
Image Classification ImageNet ResMLP-S12 Top 1 Accuracy 77.8% # 542
Number of params 15.4M # 336
Image Classification ImageNet ResMLP-36 Top 1 Accuracy 79.7% # 451
Number of params 45M # 480
Image Classification ImageNet ResMLP-24 Top 1 Accuracy 79.4% # 460
Image Classification ImageNet ReaL ResMLP-B24/8 (22k) Top 1 Accuracy 84.4% # 5
Image Classification ImageNet ReaL ResMLP-36 Accuracy 85.6% # 39
Params 45M # 37
Image Classification ImageNet ReaL ResMLP-24 Accuracy 85.3% # 41
Params 30M # 36
Image Classification ImageNet ReaL ResMLP-12 Accuracy 84.6% # 43
Params 15M # 33
Image Classification ImageNet V2 ResMLP-B24/8 22k Top 1 Accuracy 74.2 # 11
Image Classification ImageNet V2 ResMLP-S12/16 Top 1 Accuracy 66.0 # 23
Image Classification ImageNet V2 ResMLP-S24/16 Top 1 Accuracy 69.8 # 17
Image Classification ImageNet V2 ResMLP-B24/8 Top 1 Accuracy 73.4 # 13
Image Classification iNaturalist 2018 ResMLP-24 Top-1 Accuracy 64.3% # 37
Image Classification iNaturalist 2018 ResMLP-12 Top-1 Accuracy 60.2% # 41
Image Classification iNaturalist 2019 ResMLP-24 Top-1 Accuracy 72.5 # 11
Image Classification iNaturalist 2019 ResMLP-12 Top-1 Accuracy 71.0 # 13
Fine-Grained Image Classification Oxford 102 Flowers ResMLP-24 Accuracy 97.9% # 13
Fine-Grained Image Classification Oxford 102 Flowers ResMLP-12 Accuracy 97.4% # 16
Image Classification Stanford Cars ResMLP-12 Accuracy 84.6 # 18
Fine-Grained Image Classification Stanford Cars ResMLP-24 Accuracy 89.5% # 55
Fine-Grained Image Classification Stanford Cars ResMLP-12 Accuracy 84.6% # 56
Image Classification Stanford Cars ResMLP-24 Accuracy 89.5 # 13
Machine Translation WMT2014 English-French ResMLP-6 BLEU score 40.3 # 31
Machine Translation WMT2014 English-French ResMLP-12 BLEU score 40.6 # 26
Machine Translation WMT2014 English-German ResMLP-12 BLEU score 26.8 # 45
Machine Translation WMT2014 English-German ResMLP-6 BLEU score 26.4 # 47

Methods