Momentum Contrast for Unsupervised Visual Representation Learning

We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder... (read more)

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 4x) Top 1 Accuracy 68.6% # 39
Number of Params 375M # 8
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 2x) Top 1 Accuracy 65.4% # 42
Number of Params 94M # 16
Self-Supervised Image Classification ImageNet MoCo (ResNet-50) Top 1 Accuracy 60.6% # 51
Number of Params 24M # 27
Top 1 Accuracy (kNN, k=20) 47.1% # 10

Methods used in the Paper


METHOD TYPE
InfoNCE
Loss Functions
Random Grayscale
Image Data Augmentation
Random Horizontal Flip
Image Data Augmentation
ColorJitter
Image Data Augmentation
Random Resized Crop
Image Data Augmentation
FPN
Feature Extractors
RoIAlign
RoI Feature Extractors
Mask R-CNN
Instance Segmentation Models
RPN
Region Proposal
Softmax
Output Functions
RoIPool
RoI Feature Extractors
Faster R-CNN
Object Detection Models
Exponential Decay
Learning Rate Schedules
SGD with Momentum
Stochastic Optimization
MoCo
Self-Supervised Learning
Average Pooling
Pooling Operations
Residual Connection
Skip Connections
ReLU
Activation Functions
1x1 Convolution
Convolutions
Batch Normalization
Normalization
Bottleneck Residual Block
Skip Connection Blocks
Global Average Pooling
Pooling Operations
Residual Block
Skip Connection Blocks
Kaiming Initialization
Initialization
Max Pooling
Pooling Operations
Convolution
Convolutions
ResNet
Convolutional Neural Networks