Self-Supervised Learning
1688 papers with code • 10 benchmarks • 41 datasets
Self-Supervised Learning is proposed for utilizing unlabeled data with the success of supervised learning. Producing a dataset with good labels is expensive, while unlabeled data is being generated all the time. The motivation of Self-Supervised Learning is to make use of the large amount of unlabeled data. The main idea of Self-Supervised Learning is to generate the labels from unlabeled data, according to the structure or characteristics of the data itself, and then train on this unsupervised data in a supervised manner. Self-Supervised Learning is wildly used in representation learning to make a model learn the latent features of the data. This technique is often employed in computer vision, video processing and robot control.
Source: Self-supervised Point Set Local Descriptors for Point Cloud Registration
Image source: LeCun
Libraries
Use these libraries to find Self-Supervised Learning models and implementationsDatasets
Latest papers
TFPred: Learning Discriminative Representations from Unlabeled Data for Few-Label Rotating Machinery Fault Diagnosis
Recent advances in intelligent rotating machinery fault diagnosis have been enabled by the availability of massive labeled training data.
Efficient Image Pre-Training with Siamese Cropped Masked Autoencoders
In particular, SiamMAE recently introduced a Siamese network, training a shared-weight encoder from two frames of a video with a high asymmetric masking ratio (95%).
A Survey on Self-Supervised Pre-Training of Graph Foundation Models: A Knowledge-Based Perspective
Graph self-supervised learning is now a go-to method for pre-training graph foundation models, including graph neural networks, graph transformers, and more recent large language model (LLM)-based graph models.
Pose-Guided Self-Training with Two-Stage Clustering for Unsupervised Landmark Discovery
Second, motivated by the ZeroShot performance, we develop a ULD algorithm based on diffusion features using self-training and clustering which also outperforms prior methods by notable margins.
An Embarrassingly Simple Defense Against Backdoor Attacks On SSL
Using object classification as the downstream task for SSL, we demonstrate successful defense strategies that do not require re-training of the model.
Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn better representations.
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining
However, transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks.
On Pretraining Data Diversity for Self-Supervised Learning
We explore the impact of training with more diverse datasets, characterized by the number of unique samples, on the performance of self-supervised learning (SSL) under a fixed computational budget.
Diffusion-Driven Self-Supervised Learning for Shape Reconstruction and Pose Estimation
Furthermore, we introduce a pretrain-to-refine self-supervised training paradigm to train our network.
Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
On complex downstream tasks with limited data, such as fluid flow simulations and fluid-structure interactions, we found CoDA-NO to outperform existing methods on the few-shot learning task by over $36\%$.