Semi-Supervised Learning Methods
MoCo v2
Introduced by Chen et al. in Improved Baselines with Momentum Contrastive LearningMoCo v2 is an improved version of the Momentum Contrast self-supervised learning algorithm. Motivated by the findings presented in the SimCLR paper, authors:
- Replace the 1-layer fully connected layer with a 2-layer MLP head with ReLU for the unsupervised training stage.
- Include blur augmentation.
- Use cosine learning rate schedule.
These modifications enable MoCo to outperform the state-of-the-art SimCLR with a smaller batch size and fewer epochs.
Source: Improved Baselines with Momentum Contrastive Learning
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Self-Supervised Learning | 8 | 18.18% |
| Image Classification | 4 | 9.09% |
| Object Detection | 3 | 6.82% |
| Language Modelling | 2 | 4.55% |
| Machine Translation | 2 | 4.55% |
| Quantization | 2 | 4.55% |
| Self-Supervised Image Classification | 2 | 4.55% |
| Semantic Segmentation | 2 | 4.55% |
| Unsupervised Representation Learning | 2 | 4.55% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
Feedforward Network
|
Feedforward Networks | |
InfoNCE
|
Loss Functions | |
Random Gaussian Blur
|
Image Data Augmentation |