NPID++

Model Name:*

Description with Markdown (optional):

# Summary

**NPID++** (Non-Parametric Instance Discrimination) is a self-supervision approach that takes a
non-parametric classification approach. It approves upon NPID by using more negative samples and training
for more epochs.

## How do I train this model?

Get started with VISSL by trying one of the **Colab tutorial notebooks**.

- [Extracting Features from a pretrained model](https://colab.research.google.com/drive/1EXZyW65lXBryWE2ZmC0bar2996KEiBZb?usp=sharing)
- [Benchmark task: Full finetuning on ImageNet-1K](https://colab.research.google.com/drive/1m1LUa-3vIR-rxwcm0QCrefc5S6PAY874?usp=sharing)
- [Benchmark task: Linear image classification on ImageNet-1K](https://colab.research.google.com/drive/1CCuZ50BN99JcOB6VEPytVi_i2tSMd7A3?usp=sharing)
- [Large scale training (fp16, LARC, ZeRO)](https://colab.research.google.com/drive/1fvZdRNUyHxMOaxuEO34x7XeGndzTUfIW?usp=sharing)

## Citation

```BibTeX
@article{DBLP:journals/corr/abs-1912-01991,
  author    = {Ishan Misra and
               Laurens van der Maaten},
  title     = {Self-Supervised Learning of Pretext-Invariant Representations},
  journal   = {CoRR},
  volume    = {abs/1912.01991},
  year      = {2019},
  url       = {http://arxiv.org/abs/1912.01991},
  archivePrefix = {arXiv},
  eprint    = {1912.01991},
  timestamp = {Thu, 02 Jan 2020 18:08:18 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1912-01991.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```

```BibTeX
@misc{goyal2021vissl,
  author =       {Priya Goyal and Benjamin Lefaudeux and Mannat Singh and Jeremy Reizenstein and Vinicius Reis and 
                  Min Xu and and Matthew Leavitt and Mathilde Caron and Piotr Bojanowski and Armand Joulin and 
                  Ishan Misra},
  title =        {VISSL},
  howpublished = {\url{https://github.com/facebookresearch/vissl}},
  year =         {2021}
}
```

Paper:*

Code URL (optional):

ID	rn50_in1k_npid_oss
Epochs	800
Layers	50
Classes	1000
Momentum	0.9
Negatives	32000
Weight Decay	0.0001
Width Multiplier	1
NCE Loss Temperature	0.07

Attached motifs:

1X1 CONVOLUTION

RELU

BATCH NORMALIZATION

BOTTLENECK RESIDUAL BLOCK

RESIDUAL CONNECTION

GLOBAL AVERAGE POOLING

CONVOLUTION

RESIDUAL BLOCK

SOFTMAX

MAX POOLING

1X1 CONVOLUTION

Epochs	800
Layers	50
Classes	1000
Momentum	0.9
Negatives	32000
Weight Decay	0.0001
Width Multiplier	2
NCE Loss Temperature	0.07

Attached motifs:

1X1 CONVOLUTION

RELU

BATCH NORMALIZATION

BOTTLENECK RESIDUAL BLOCK

RESIDUAL CONNECTION

GLOBAL AVERAGE POOLING

CONVOLUTION

RESIDUAL BLOCK

SOFTMAX

MAX POOLING

1X1 CONVOLUTION

BENCHMARK	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
ImageNet	NPID++ ResNet-50-w2 (32k negatives, 800 epochs, cosine LR)	Top 1 Accuracy	62.73%	# 301
ImageNet	NPID++ ResNet-50 (32k negatives, 800 epochs, cosine LR)	Top 1 Accuracy	56.68%	# 305

facebookresearch / vissl

Summary

How do I train this model?

Citation

Results

Image Classification on ImageNet

Image Classification

Training Techniques	NPID++, NPID, Weight Decay, SGD with Momentum, Cosine Annealing
Architecture	1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID	rn50_in1k_npid_oss
Epochs	800
Layers	50
Classes	1000
Momentum	0.9
Negatives	32000
Weight Decay	0.0001
Width Multiplier	1
NCE Loss Temperature	0.07
SHOW MORE
SHOW LESS