RegNetX

Last updated on Feb 14, 2021

regnetx_002

Parameters 3 Million
FLOPs 255 Million
File Size 10.36 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_002
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 1024
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_004

Parameters 5 Million
FLOPs 511 Million
File Size 19.88 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_004
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 1024
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_006

Parameters 6 Million
FLOPs 772 Million
File Size 23.81 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_006
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 1024
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_008

Parameters 7 Million
FLOPs 1 Billion
File Size 27.88 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_008
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 1024
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_016

Parameters 9 Million
FLOPs 2 Billion
File Size 35.27 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_016
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 1024
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_032

Parameters 15 Million
FLOPs 4 Billion
File Size 58.66 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_032
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_040

Parameters 22 Million
FLOPs 5 Billion
File Size 84.73 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_040
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_064

Parameters 26 Million
FLOPs 8 Billion
File Size 100.31 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_064
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_080

Parameters 40 Million
FLOPs 10 Billion
File Size 151.37 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_080
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_120

Parameters 46 Million
FLOPs 16 Billion
File Size 176.30 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_120
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_160

Parameters 54 Million
FLOPs 20 Billion
File Size 207.54 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_160
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 512
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
regnetx_320

Parameters 108 Million
FLOPs 41 Billion
File Size 411.95 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Batch Normalization, Convolution, Dense Connections, Global Average Pooling, Grouped Convolution, ReLU
ID regnetx_320
Epochs 100
Crop Pct 0.875
Momentum 0.9
Batch Size 256
Image Size 224
Weight Decay 0.00005
Interpolation bicubic
SHOW MORE
SHOW LESS
README.md

Summary

RegNetX is a convolutional network design space with simple, regular models with parameters: depth $d$, initial width $w_{0} > 0$, and slope $w_{a} > 0$, and generates a different block width $u_{j}$ for each block $j < d$. The key restriction for the RegNet types of model is that there is a linear parameterisation of block widths (the design space only contains models with this linear structure):

$$ u_{j} = w_{0} + w_{a}\cdot{j} $$

For RegNetX we have additional restrictions: we set $b = 1$ (the bottleneck ratio), $12 \leq d \leq 28$, and $w_{m} \geq 2$ (the width multiplier).

How do I load this model?

To load a pretrained model:

import timm
m = timm.create_model('regnetx_002', pretrained=True)
m.eval()

Replace the model name with the variant you want to use, e.g. regnetx_002. You can find the IDs in the model summaries at the top of this page.

How do I train this model?

You can follow the timm recipe scripts for training a new model afresh.

Citation

@misc{radosavovic2020designing,
      title={Designing Network Design Spaces}, 
      author={Ilija Radosavovic and Raj Prateek Kosaraju and Ross Girshick and Kaiming He and Piotr Dollár},
      year={2020},
      eprint={2003.13678},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Results

Image Classification on ImageNet

Image Classification on ImageNet
MODEL TOP 1 ACCURACY TOP 5 ACCURACY
regnetx_320 80.25% 95.03%
regnetx_160 79.84% 94.82%
regnetx_120 79.61% 94.73%
regnetx_080 79.21% 94.55%
regnetx_064 79.06% 94.47%
regnetx_040 78.48% 94.25%
regnetx_032 78.15% 94.09%
regnetx_016 76.95% 93.43%
regnetx_008 75.05% 92.34%
regnetx_006 73.84% 91.68%
regnetx_004 72.39% 90.82%
regnetx_002 68.75% 88.56%