NPID

Model Name:*

Description with Markdown (optional):

# Summary

**NPID** (Non-Parametric Instance Discrimination) is a self-supervision approach that takes a
non-parametric classification approach. Noise contrastive estimation is used to learn representations. Specifically,
distances (similarity) between instances are calculated directly from the features in a
non-parametric way.

## How do I train this model?

Get started with VISSL by trying one of the **Colab tutorial notebooks**.

- [Extracting Features from a pretrained model](https://colab.research.google.com/drive/1EXZyW65lXBryWE2ZmC0bar2996KEiBZb?usp=sharing)
- [Benchmark task: Full finetuning on ImageNet-1K](https://colab.research.google.com/drive/1m1LUa-3vIR-rxwcm0QCrefc5S6PAY874?usp=sharing)
- [Benchmark task: Linear image classification on ImageNet-1K](https://colab.research.google.com/drive/1CCuZ50BN99JcOB6VEPytVi_i2tSMd7A3?usp=sharing)
- [Large scale training (fp16, LARC, ZeRO)](https://colab.research.google.com/drive/1fvZdRNUyHxMOaxuEO34x7XeGndzTUfIW?usp=sharing)

## Citation

```BibTeX
@article{DBLP:journals/corr/abs-1805-01978,
  author    = {Zhirong Wu and
               Yuanjun Xiong and
               Stella X. Yu and
               Dahua Lin},
  title     = {Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination},
  journal   = {CoRR},
  volume    = {abs/1805.01978},
  year      = {2018},
  url       = {http://arxiv.org/abs/1805.01978},
  archivePrefix = {arXiv},
  eprint    = {1805.01978},
  timestamp = {Wed, 12 Aug 2020 11:07:47 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1805-01978.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```

```BibTeX
@misc{goyal2021vissl,
  author =       {Priya Goyal and Benjamin Lefaudeux and Mannat Singh and Jeremy Reizenstein and Vinicius Reis and 
                  Min Xu and and Matthew Leavitt and Mathilde Caron and Piotr Bojanowski and Armand Joulin and 
                  Ishan Misra},
  title =        {VISSL},
  howpublished = {\url{https://github.com/facebookresearch/vissl}},
  year =         {2021}
}
```

Paper:*

Code URL (optional):

ID	rn50_in1k_npid_200ep_4kneg
Epochs	200
Layers	50
Classes	1000
Momentum	0.9
Negatives	4000
Weight Decay	0.0001
Width Multiplier	1
NCE Loss Temperature	0.07

Attached motifs:

1X1 CONVOLUTION

RELU

BATCH NORMALIZATION

BOTTLENECK RESIDUAL BLOCK

RESIDUAL CONNECTION

GLOBAL AVERAGE POOLING

CONVOLUTION

RESIDUAL BLOCK

SOFTMAX

MAX POOLING

1X1 CONVOLUTION

ID	rn50_in1k_npid_oss
Epochs	200
Layers	50
Classes	1000
Momentum	0.9
Negatives	4096
Batch Size	256
Width Multiplier	1
NCE Loss Temperature	0.07

Attached motifs:

1X1 CONVOLUTION

RELU

BATCH NORMALIZATION

BOTTLENECK RESIDUAL BLOCK

RESIDUAL CONNECTION

GLOBAL AVERAGE POOLING

CONVOLUTION

RESIDUAL BLOCK

SOFTMAX

MAX POOLING

1X1 CONVOLUTION

BENCHMARK	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
ImageNet	NPID ResNet-50 (ImageNet-1K, official)	Top 1 Accuracy	54.99%	# 307
ImageNet	NPID ResNet-50 (4k negatives, 200 epochs)	Top 1 Accuracy	52.73%	# 311

facebookresearch / vissl

Summary

How do I train this model?

Citation

Results

Image Classification on ImageNet

Image Classification

Training Techniques	NPID, Weight Decay, SGD with Momentum
Architecture	1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID	rn50_in1k_npid_200ep_4kneg
Epochs	200
Layers	50
Classes	1000
Momentum	0.9
Negatives	4000
Weight Decay	0.0001
Width Multiplier	1
NCE Loss Temperature	0.07
SHOW MORE
SHOW LESS