SSL ResNet

Last updated on Feb 14, 2021

ssl_resnet18

Parameters 12 Million
FLOPs 2 Billion
File Size 44.64 MB
Training Data ImageNet
Training Resources 64x GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID ssl_resnet18
LR 0.0015
Epochs 30
Layers 18
Crop Pct 0.875
Batch Size 1536
Image Size 224
Weight Decay 0.0001
Interpolation bilinear
SHOW MORE
SHOW LESS
ssl_resnet50

Parameters 26 Million
FLOPs 5 Billion
File Size 97.73 MB
Training Data ImageNet
Training Resources 64x GPUs
Training Time

Training Techniques SGD with Momentum, Weight Decay
Architecture 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID ssl_resnet50
LR 0.0015
Epochs 30
Layers 50
Crop Pct 0.875
Batch Size 1536
Image Size 224
Weight Decay 0.0001
Interpolation bilinear
SHOW MORE
SHOW LESS
README.md

Summary

Residual Networks, or ResNets, learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. Instead of hoping each few stacked layers directly fit a desired underlying mapping, residual nets let these layers fit a residual mapping. They stack residual blocks ontop of each other to form network: e.g. a ResNet-50 has fifty layers using these blocks.

The model in this collection utilises semi-supervised learning to improve the performance of the model. The approach brings important gains to standard architectures for image, video and fine-grained classification.

Please note the CC-BY-NC 4.0 license on theses weights, non-commercial use only.

How do I load this model?

To load a pretrained model:

import timm
m = timm.create_model('ssl_resnet50', pretrained=True)
m.eval()

Replace the model name with the variant you want to use, e.g. ssl_resnet50. You can find the IDs in the model summaries at the top of this page.

How do I train this model?

You can follow the timm recipe scripts for training a new model afresh.

Citation

@article{DBLP:journals/corr/abs-1905-00546,
  author    = {I. Zeki Yalniz and
               Herv{\'{e}} J{\'{e}}gou and
               Kan Chen and
               Manohar Paluri and
               Dhruv Mahajan},
  title     = {Billion-scale semi-supervised learning for image classification},
  journal   = {CoRR},
  volume    = {abs/1905.00546},
  year      = {2019},
  url       = {http://arxiv.org/abs/1905.00546},
  archivePrefix = {arXiv},
  eprint    = {1905.00546},
  timestamp = {Mon, 28 Sep 2020 08:19:37 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1905-00546.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

Results

Image Classification on ImageNet

Image Classification
BENCHMARK MODEL METRIC NAME METRIC VALUE GLOBAL RANK
ImageNet ssl_resnet50 Top 1 Accuracy 79.24% # 128
Top 5 Accuracy 94.83% # 128
ImageNet ssl_resnet18 Top 1 Accuracy 72.62% # 255
Top 5 Accuracy 91.42% # 255