Representation Learning with Contrastive Predictive Coding

10 Jul 2018Aaron van den OordYazhe LiOriol Vinyals

While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Self-Supervised Image Classification ImageNet CPC (ResNet-V2) Top 1 Accuracy 48.7% # 21
Self-Supervised Image Classification ImageNet CPC (ResNet-V2) Top 5 Accuracy 73.6% # 15
Semi-Supervised Image Classification ImageNet - 10% labeled data CPC Top 5 Accuracy 84.88% # 7
Semi-Supervised Image Classification ImageNet - 1% labeled data CPC Top 5 Accuracy 64.03% # 5