Language Models
UL2
Introduced by Tay et al. in UL2: Unifying Language Learning ParadigmsUL2 is a unified framework for pretraining models that are universally effective across datasets and setups. UL2 uses Mixture-of-Denoisers (MoD), a pre-training objective that combines diverse pre-training paradigms together. UL2 introduces a notion of mode switching, wherein downstream fine-tuning is associated with specific pre-training schemes.
Source: UL2: Unifying Language Learning Paradigms
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Question Answering | 5 | 16.67% |
| Retrieval | 3 | 10.00% |
| Language Modelling | 2 | 6.67% |
| Relation Extraction | 2 | 6.67% |
| Natural Language Inference | 2 | 6.67% |
| Natural Language Understanding | 1 | 3.33% |
| Named Entity Recognition (NER) | 1 | 3.33% |
| NER | 1 | 3.33% |
| Semantic Parsing | 1 | 3.33% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
