Language Models
mBART
Introduced by Liu et al. in Multilingual Denoising Pre-training for Neural Machine TranslationmBART is a sequence-to-sequence denoising auto-encoder pre-trained on large-scale monolingual corpora in many languages using the BART objective. The input texts are noised by masking phrases and permuting sentences, and a single Transformer model is learned to recover the texts. Different from other pre-training approaches for machine translation, mBART pre-trains a complete autoregressive Seq2Seq model. mBART is trained once for all languages, providing a set of parameters that can be fine-tuned for any of the language pairs in both supervised and unsupervised settings, without any task-specific or language-specific modifications or initialization schemes.
Source: Multilingual Denoising Pre-training for Neural Machine Translation
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Machine Translation | 25 | 23.81% |
| Denoising | 8 | 7.62% |
| Text Generation | 6 | 5.71% |
| NMT | 5 | 4.76% |
| Abstractive Text Summarization | 5 | 4.76% |
| Language Modelling | 4 | 3.81% |
| Cross-Lingual Transfer | 4 | 3.81% |
| Text Summarization | 4 | 3.81% |
| Natural Language Understanding | 3 | 2.86% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
