EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance... (read more)

PDF Abstract

Results from the Paper


Ranked #2 on Fine-Grained Image Classification on Birdsnap (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Fine-Grained Image Classification Birdsnap EfficientNet-B7 Accuracy 84.3% # 2
Image Classification CIFAR-10 EfficientNet-B7 Percentage correct 98.9 # 9
PARAMS 64M # 13
Image Classification CIFAR-100 EfficientNet-B7 Percentage correct 91.7 # 5
PARAMS 64M # 10
Fine-Grained Image Classification FGVC Aircraft EfficientNet-B7 Accuracy 92.9% # 12
Image Classification Flowers-102 EfficientNet-B7 Accuracy 98.8% # 6
Fine-Grained Image Classification Food-101 EfficientNet-B7 Accuracy 93.0 # 4
Image Classification ImageNet EfficientNet-B7 Top 1 Accuracy 84.4% # 32
Top 5 Accuracy 97.1% # 17
Number of params 66M # 37
Image Classification ImageNet EfficientNet-B5 Top 1 Accuracy 83.3% # 44
Top 5 Accuracy 96.7% # 21
Number of params 30M # 59
Image Classification ImageNet EfficientNet-B2 Top 1 Accuracy 79.8% # 92
Top 5 Accuracy 94.9% # 49
Number of params 9.2M # 80
Image Classification ImageNet EfficientNet-B3 Top 1 Accuracy 81.1% # 72
Top 5 Accuracy 95.5% # 40
Number of params 12M # 76
Image Classification ImageNet EfficientNet-B1 Top 1 Accuracy 78.8% # 106
Top 5 Accuracy 94.4% # 66
Number of params 7.8M # 83
Image Classification ImageNet EfficientNet-B0 Top 1 Accuracy 76.3% # 142
Top 5 Accuracy 93.2% # 85
Number of params 5.3M # 94
Image Classification ImageNet EfficientNet-B6 Top 1 Accuracy 84% # 36
Top 5 Accuracy 96.9% # 19
Number of params 43M # 48
Image Classification ImageNet EfficientNet-B4 Top 1 Accuracy 82.6% # 52
Top 5 Accuracy 96.3% # 26
Number of params 19M # 73
Fine-Grained Image Classification Oxford-IIIT Pets EfficientNet-B7 Accuracy 95.4% # 6
Fine-Grained Image Classification Stanford Cars EfficientNet-B7 Accuracy 94.7% # 11

Methods used in the Paper


METHOD TYPE
Tanh Activation
Activation Functions
Global Average Pooling
Pooling Operations
Bottleneck Residual Block
Skip Connection Blocks
Max Pooling
Pooling Operations
Kaiming Initialization
Initialization
Residual Connection
Skip Connections
Residual Block
Skip Connection Blocks
ResNet
Convolutional Neural Networks
Depthwise Convolution
Convolutions
Pointwise Convolution
Convolutions
Softmax
Output Functions
Depthwise Separable Convolution
Convolutions
ReLU
Activation Functions
Sigmoid Activation
Activation Functions
LSTM
Recurrent Neural Networks
Squeeze-and-Excitation Block
Image Model Blocks
Inverted Residual Block
Skip Connection Blocks
EfficientNet
Image Models
Weight Decay
Regularization
RMSProp
Stochastic Optimization
Batch Normalization
Normalization
Average Pooling
Pooling Operations
Dense Connections
Feedforward Networks
1x1 Convolution
Convolutions
Convolution
Convolutions
Dropout
Regularization
AutoAugment
Image Data Augmentation
Stochastic Depth
Regularization
Swish
Activation Functions