Two New Datasets for Italian-Language Abstractive Text Summarization
Text summarization aims to produce a short summary containing relevant parts from a given text. Due to the lack of data for abstractive summarization on low-resource languages such as Italian, we propose two new original datasets collected from two Italian news websites with multi-sentence summaries and corresponding articles, and from a dataset obtained by machine translation of a Spanish summarization dataset. These two datasets are currently the only two available in Italian for this task. To evaluate the quality of these two datasets, we used them to train a T5-base model and an mBART model, obtaining good results with both. To better evaluate the results obtained, we also compared the same models trained on automatically translated datasets, and the resulting summaries in the same training language, with the automatically translated summaries, which demonstrated the superiority of the models obtained from the proposed datasets.
PDF AbstractCode
Spaces
Datasets
Introduced in the Paper:
Abstractive Text Summarization from Fanpage Abstractive Text Summarization from Il Post MLSum-itUsed in the Paper:
C4
CNN/Daily Mail
mC4
MLSUM
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Abstractive Text Summarization | Abstractive Text Summarization from Fanpage | mBART | ROUGE-1 | 36.50 | # 2 | |
| Abstractive Text Summarization | Abstractive Text Summarization from Fanpage | IT5 | ROUGE-1 | 33.83 | # 6 | |
| Abstractive Text Summarization | Abstractive Text Summarization from Il Post | IT5 | ROUGE-1 | 33.78 | # 5 | |
| Abstractive Text Summarization | Abstractive Text Summarization from Il Post | Pegasus-CNN/DM (eng-it translation) | ROUGE-1 | 23.96 | # 7 | |
| Abstractive Text Summarization | Abstractive Text Summarization from Il Post | Pegasus-XSum (eng-it translation) | ROUGE-1 | 21.03 | # 8 | |
| Abstractive Text Summarization | Abstractive Text Summarization from Il Post | mBART | ROUGE-1 | 38.91 | # 1 | |
| Abstractive Text Summarization | MLSum-it | IT5 | rouge1 | 19.29 | # 2 | |
| Abstractive Text Summarization | MLSum-it | Pegasus-CNN/DM (eng-it translation) | rouge1 | 16.97 | # 3 | |
| Abstractive Text Summarization | MLSum-it | Pegasus-XSum (eng-it translation) | rouge1 | 15.17 | # 4 | |
| Abstractive Text Summarization | MLSum-it | mBART | rouge1 | 19.35 | # 1 |
