Self-Supervised Learning
Mirror-BERT
Introduced by Liu et al. in Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence EncodersMirror-BERT converts pretrained language models into effective universal text encoders without any supervision, in 20-30 seconds. It is an extremely simple, fast, and effective contrastive learning technique. It relies on fully identical or slightly modified string pairs as positive (i.e., synonymous) fine-tuning examples, and aims to maximise their similarity during identity fine-tuning.
Source: Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Sentence | 3 | 15.00% |
| Entity Linking | 2 | 10.00% |
| Language Modelling | 2 | 10.00% |
| Semantic Textual Similarity | 2 | 10.00% |
| Natural Language Understanding | 1 | 5.00% |
| Bilingual Lexicon Induction | 1 | 5.00% |
| Cross-Lingual Entity Linking | 1 | 5.00% |
| Knowledge Base Construction | 1 | 5.00% |
| Language Modeling | 1 | 5.00% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
