Self-Supervised Learning

Unsupervised knowledge distillation from a pretrained language model to itself, by alternating between its bi- and cross-encoder forms.

Source: Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Language Modelling 1 33.33%
Paraphrase Identification 1 33.33%
Semantic Textual Similarity 1 33.33%

Categories