AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization
Like most natural language understanding and generation tasks, state-of-the-art models for summarization are transformer-based sequence-to-sequence architectures that are pretrained on large corpora. While most existing models focused on English, Arabic remained understudied. In this paper we propose AraBART, the first Arabic model in which the encoder and the decoder are pretrained end-to-end, based on BART. We show that AraBART achieves the best performance on multiple abstractive summarization datasets, outperforming strong baselines including a pretrained Arabic BERT-based model and multilingual mBART and mT5 models.
PDF Abstract
Datasets
Add Datasets
introduced or used in this paper
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
Adafactor •
Adam •
Attention Dropout •
BART •
BPE •
Dense Connections •
Dropout •
GELU •
GLU •
Inverse Square Root Schedule •
Layer Normalization •
Linear Layer •
mBART •
mT5 •
Multi-Head Attention •
Residual Connection •
Scaled Dot-Product Attention •
SentencePiece •
Softmax •
T5
