Self-Supervised Learning

Edit

General • 47 methods

Self-Supervised Learning refers to a category of methods where we learn representations in a self-supervised way (i.e without labels). These methods generally involve a pretext task that is solved to learn a good representation and a loss function to learn with. Below you can find a continuously updating list of self-supervised methods.

Methods

Add a Method

Method	Year	Papers
Inpainting Context Encoders: Feature Learning by Inpainting	2016	876
MAE Masked Autoencoders Are Scalable Vision Learners	2021	595
SimCLR A Simple Framework for Contrastive Learning of Visual Representations	2020	229
Colorization Colorful Image Colorization	2016	198
DINO Emerging Properties in Self-Supervised Vision Transformers	2021	158
MoCo Momentum Contrast for Unsupervised Visual Representation Learning	2019	142
Jigsaw Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles	2016	133
BYOL Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning	2020	119
Contrastive Predictive Coding Representation Learning with Contrastive Predictive Coding	2018	108
Barlow Twins Barlow Twins: Self-Supervised Learning via Redundancy Reduction	2021	67
SwAV Unsupervised Learning of Visual Features by Contrasting Cluster Assignments	2020	49
MoCo v2 Improved Baselines with Momentum Contrastive Learning	2020	30
COLA Contrastive Learning of General-Purpose Audio Representations	2020	29
SEER Self-supervised Pretraining of Visual Features in the Wild	2021	15
BiGAN Adversarial Feature Learning	2016	14
NCL Neighborhood Contrastive Learning Applied to Online Patient Monitoring	2021	13
DeepCluster Deep Clustering for Unsupervised Learning of Visual Features	2018	10
ReLIC Representation Learning via Invariant Causal Mechanisms	2020	10
CMCL UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning	2020	10
Dense Contrastive Learning Dense Contrastive Learning for Self-Supervised Visual Pre-Training	2020	9
PIRL Self-Supervised Learning of Pretext-Invariant Representations	2019	8
SSDS OCTAve: 2D en face Optical Coherence Tomography Angiography Vessel Segmentation in Weakly-Supervised Learning with Locality Augmentation	2022	8
Mirror-BERT Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders	2021	6
MARLIN MARLIN: Masked Autoencoder for facial video Representation LearnINg	2022	6
M2D Masked Modeling Duo: Towards a Universal Audio Pre-training Framework	2024	6
Contrastive Multiview Coding Contrastive Multiview Coding	2019	5
NNCLR With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations	2021	5
BigBiGAN Large Scale Adversarial Representation Learning	2019	4
RotNet RotNet: Fast and Scalable Estimation of Stellar Rotation Periods Using Convolutional Neural Networks	2020	4
NPID Unsupervised Feature Learning via Non-Parametric Instance Discrimination	2018	4
Graph Contrastive Coding GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training	2020	4
DeCLUTR DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations	2020	3
CVRL Spatiotemporal Contrastive Video Representation Learning	2020	3
CRISS Cross-lingual Retrieval for Iterative Self-Supervised Training	2020	3
IMGEP Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning	2017	2
PASE+ Multi-task self-supervised learning for Robust Speech Recognition	2020	2
MoBY Self-Supervised Learning with Swin Transformers	2021	2
CPC v2 Data-Efficient Image Recognition with Contrastive Predictive Coding	2019	1
ALP-GMM Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments	2019	1
ClusterFit ClusterFit: Improving Generalization of Visual Representations	2019	1
NPID++ Self-Supervised Learning of Pretext-Invariant Representations	2019	1
VC R-CNN Visual Commonsense R-CNN	2020	1
ParamCrop ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning	2021	1
Trans-Encoder Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations	2021	1
Magnification Prior Contrastive Similarity Magnification Prior: A Self-Supervised Method for Learning Representations on Breast Cancer Histopathological Images	2022	1
Internet Explorer Internet Explorer: Targeted Representation Learning on the Open Web	2023	1

Self-Supervised Learning Edit

Methods Add a Method

Self-Supervised Learning

Edit

Methods

Add a Method