| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | linear |
| SHOW MORE |
| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | Dense Connections, 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | MLP |
| SHOW MORE |
| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | linear |
| SHOW MORE |
| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | Dense Connections, 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | MLP |
| SHOW MORE |
| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | Linear |
| SHOW MORE |
| Training Techniques | PIRL, Weight Decay, SGD with Momentum, Cosine Annealing |
|---|---|
| Architecture | Dense Connections, 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax |
| Head | MLP |
| SHOW MORE |
Pretext-Invariant Representation Learning (PIRL, pronounced as “pearl”) learns invariant representations based on pretext tasks. PIRL is used with a commonly used pretext task that involves solving jigsaw puzzles. Specifically, PIRL constructs image representations that are similar to the representation of transformed versions of the same image and different from the representations of other images.
Get started with VISSL by trying one of the Colab tutorial notebooks.
@inproceedings{misra2020pirl,
title={Self-Supervised Learning of Pretext-Invariant Representations},
author={Misra, Ishan and van der Maaten, Laurens},
booktitle={CVPR},
year={2020}
}
Image Classification on ImageNet
| MODEL | TOP 1 ACCURACY |
|---|---|
| PIRL ResNet-50-w2 (400 epochs, MLP head) | 70.9% |
| PIRL ResNet-50 (800 epochs, MLP head) | 69.9% |
| PIRL ResNet-50-w2 (400 epochs, linear head) | 69.3% |
| PIRL ResNet-50 (200 epochs, MLP head) | 65.8% |
| PIRL ResNet-50 (800 epochs, linear head) | 64.29% |
| PIRL ResNet-50 (200 epochs, linear head) | 62.9% |
