Self-Supervised Learning
MoCo v2
Introduced by Chen et al. in Improved Baselines with Momentum Contrastive LearningMoCo v2 is an improved version of the Momentum Contrast self-supervised learning algorithm. Motivated by the findings presented in the SimCLR paper, authors:
- Replace the 1-layer fully connected layer with a 2-layer MLP head with ReLU for the unsupervised training stage.
- Include blur augmentation.
- Use cosine learning rate schedule.
These modifications enable MoCo to outperform the state-of-the-art SimCLR with a smaller batch size and fewer epochs.
Source: Improved Baselines with Momentum Contrastive Learning
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Self-Supervised Learning | 16 | 20.25% |
| Image Classification | 8 | 10.13% |
| Semantic Segmentation | 6 | 7.59% |
| Object Detection | 5 | 6.33% |
| Classification | 3 | 3.80% |
| Language Modeling | 2 | 2.53% |
| Language Modelling | 2 | 2.53% |
| Instance Segmentation | 2 | 2.53% |
| Object | 2 | 2.53% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
Feedforward Network
|
Feedforward Networks | |
InfoNCE
|
Loss Functions | |
Random Gaussian Blur
|
Image Data Augmentation |
