Improved Baselines with Momentum Contrastive Learning

9 Mar 2020  ·  Xinlei Chen, Haoqi Fan, Ross Girshick, Kaiming He ·

Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Self-Supervised Image Classification ImageNet MoCo v2 (ResNet-50) Top 1 Accuracy 71.1% # 69
Top 5 Accuracy 90.1% # 22
Number of Params 24M # 34
Contrastive Learning imagenet-1k ResNet50 ImageNet Top-1 Accuracy 71.1 # 3
Image Classification Places205 MoCo v2 Top 1 Accuracy 52.9 # 14
Self-Supervised Person Re-Identification SYSU-30k MoCo v2 Rank-1 11.6 # 3
Person Re-Identification SYSU-30k MoCo v2 (self-supervised) Rank-1 11.6 # 6