ConvMLP: Hierarchical Convolutional MLPs for Vision

9 Sep 2021  ·  Jiachen Li, Ali Hassani, Steven Walton, Humphrey Shi ·

MLP-based architectures, which consist of a sequence of consecutive multi-layer perceptron blocks, have recently been found to reach comparable results to convolutional and transformer-based methods. However, most adopt spatial MLPs which take fixed dimension inputs, therefore making it difficult to apply them to downstream tasks, such as object detection and semantic segmentation. Moreover, single-stage designs further limit performance in other computer vision tasks and fully connected layers bear heavy computation. To tackle these problems, we propose ConvMLP: a hierarchical Convolutional MLP for visual recognition, which is a light-weight, stage-wise, co-design of convolution layers, and MLPs. In particular, ConvMLP-S achieves 76.8% top-1 accuracy on ImageNet-1k with 9M parameters and 2.4G MACs (15% and 19% of MLP-Mixer-B/16, respectively). Experiments on object detection and semantic segmentation further show that visual representation learned by ConvMLP can be seamlessly transferred and achieve competitive results with fewer parameters. Our code and pre-trained models are publicly available at https://github.com/SHI-Labs/Convolutional-MLPs.

PDF Abstract

Results from the Paper


Ranked #7 on Image Classification on Flowers-102 (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Semantic Segmentation ADE20K ConvMLP-L Validation mIoU 40 # 172
Semantic Segmentation ADE20K ConvMLP-S Validation mIoU 35.8 # 178
Semantic Segmentation ADE20K ConvMLP-M Validation mIoU 38.6 # 173
Image Classification CIFAR-10 ConvMLP-S Percentage correct 98 # 51
Image Classification CIFAR-10 ConvMLP-L Percentage correct 98.6 # 28
Image Classification CIFAR-10 ConvMLP-M Percentage correct 98.6 # 28
Image Classification CIFAR-100 ConvMLP-S Percentage correct 87.4 # 47
Image Classification CIFAR-100 ConvMLP-M Percentage correct 89.1 # 34
Image Classification CIFAR-100 ConvMLP-L Percentage correct 88.6 # 36
Image Classification Flowers-102 ConvMLP-S Accuracy 99.5 # 7
Image Classification Flowers-102 ConvMLP-L Accuracy 99.5 # 7
Image Classification ImageNet ConvMLP-S Top 1 Accuracy 76.8 # 700
Number of params 9M # 378
Image Classification ImageNet ConvMLP-M Top 1 Accuracy 79% # 609
Number of params 17.4M # 425
Image Classification ImageNet ConvMLP-L Top 1 Accuracy 80.2% # 549
Number of params 42.7M # 582

Methods