Transformers
PEGASUS
Introduced by Zhang et al. in PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive SummarizationPEGASUS proposes a transformer-based model for abstractive summarization. It uses a special self-supervised pre-training objective called gap-sentences generation (GSG) that's designed to perform well on summarization-related downstream tasks. As reported in the paper, "both GSG and MLM are applied simultaneously to this example as pre-training objectives. Originally there are three sentences. One sentence is masked with [MASK1] and used as target generation text (GSG). The other two sentences remain in the input, but some tokens are randomly masked by [MASK2]."
Source: PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Abstractive Text Summarization | 22 | 22.22% |
| Text Summarization | 12 | 12.12% |
| Document Summarization | 6 | 6.06% |
| Decoder | 5 | 5.05% |
| Multi-Document Summarization | 4 | 4.04% |
| Sentence | 4 | 4.04% |
| Domain Adaptation | 3 | 3.03% |
| Text Generation | 3 | 3.03% |
| Active Learning | 3 | 3.03% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
