Fine-tune BERT for Extractive Summarization

arXiv 2019 Yang Liu

BERT (Devlin et al., 2018), a pre-trained Transformer (Vaswani et al., 2017) model, has achieved ground-breaking performance on multiple NLP tasks. In this paper, we describe BERTSUM, a simple variant of BERT, for extractive summarization... (read more)

PDF Abstract

Evaluation Results from the Paper


#3 best model for Document Summarization on CNN / Daily Mail (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
COMPARE
Document Summarization CNN / Daily Mail BERTSUM+Transformer ROUGE-1 43.25 # 3
Document Summarization CNN / Daily Mail BERTSUM+Transformer ROUGE-2 20.24 # 4
Document Summarization CNN / Daily Mail BERTSUM+Transformer ROUGE-L 39.63 # 4