ResNet Semi-supervised

Model Name:*

Description with Markdown (optional):

# Summary

**ResNet SSL** is a model that uses semi-supervised learning to learn image representations.
 It utilises a  pipeline, based on a teacher/student paradigm, that leverages a large
collection of unlabelled images (up to 1 billion).  Even in the case of self-training – when teacher and student models are the same - the sheer scale of unlabeled data enables significant gains. Performance is sensitive
to several factors like strength of initial (teacher) model for ranking, scale and nature of unlabeled data, relationship between teacher and final model, etc.

## How do I train this model?

Get started with VISSL by trying one of the **Colab tutorial notebooks**.

- [Extracting Features from a pretrained model](https://colab.research.google.com/drive/1EXZyW65lXBryWE2ZmC0bar2996KEiBZb?usp=sharing)
- [Benchmark task: Full finetuning on ImageNet-1K](https://colab.research.google.com/drive/1m1LUa-3vIR-rxwcm0QCrefc5S6PAY874?usp=sharing)
- [Benchmark task: Linear image classification on ImageNet-1K](https://colab.research.google.com/drive/1CCuZ50BN99JcOB6VEPytVi_i2tSMd7A3?usp=sharing)
- [Large scale training (fp16, LARC, ZeRO)](https://colab.research.google.com/drive/1fvZdRNUyHxMOaxuEO34x7XeGndzTUfIW?usp=sharing)

## Citation

```BibTeX
@article{DBLP:journals/corr/abs-1905-00546,
  author    = {I. Zeki Yalniz and
               Herv{\'{e}} J{\'{e}}gou and
               Kan Chen and
               Manohar Paluri and
               Dhruv Mahajan},
  title     = {Billion-scale semi-supervised learning for image classification},
  journal   = {CoRR},
  volume    = {abs/1905.00546},
  year      = {2019},
  url       = {http://arxiv.org/abs/1905.00546},
  archivePrefix = {arXiv},
  eprint    = {1905.00546},
  timestamp = {Mon, 28 Sep 2020 08:19:37 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1905-00546.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```

```BibTeX
@misc{goyal2021vissl,
  author =       {Priya Goyal and Benjamin Lefaudeux and Mannat Singh and Jeremy Reizenstein and Vinicius Reis and 
                  Min Xu and and Matthew Leavitt and Mathilde Caron and Piotr Bojanowski and Armand Joulin and 
                  Ishan Misra},
  title =        {VISSL},
  howpublished = {\url{https://github.com/facebookresearch/vissl}},
  year =         {2021}
}
```

Paper:*

Code URL (optional):

ID	resnet50_ssl
Layers	50
Momentum	0.9
Batch Size	1536
Weight Decay	0.0001
Width Multiplier	1

Attached motifs:

1X1 CONVOLUTION

RELU

BATCH NORMALIZATION

BOTTLENECK RESIDUAL BLOCK

RESIDUAL CONNECTION

GLOBAL AVERAGE POOLING

CONVOLUTION

RESIDUAL BLOCK

SOFTMAX

MAX POOLING

1X1 CONVOLUTION

facebookresearch / vissl

Summary

How do I train this model?

Citation

Results

Image Classification on ImageNet

Image Classification

Training Techniques	Weight Decay, SGD with Momentum
Architecture	1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID	resnet50_ssl
Layers	50
Momentum	0.9
Batch Size	1536
Weight Decay	0.0001
Width Multiplier	1
SHOW MORE
SHOW LESS