Learn to Copy from the Copying History: Correlational Copy Network for Abstractive Summarization

The copying mechanism has had considerable success in abstractive summarization, facilitating models to directly copy words from the input text to the output summary. Existing works mostly employ encoder-decoder attention, which applies copying at each time step independently of the former ones. However, this may sometimes lead to incomplete copying. In this paper, we propose a novel copying scheme named Correlational Copying Network (CoCoNet) that enhances the standard copying mechanism by keeping track of the copying history. It thereby takes advantage of prior copying distributions and, at each time step, explicitly encourages the model to copy the input word that is relevant to the previously copied one. In addition, we strengthen CoCoNet through pre-training with suitable corpora that simulate the copying behaviors. Experimental results show that CoCoNet can copy more accurately and achieves new state-of-the-art performances on summarization benchmarks, including CNN/DailyMail for news summarization and SAMSum for dialogue summarization. The code and checkpoint will be publicly available.

PDF Abstract

Results from the Paper


Ranked #11 on Abstractive Text Summarization on CNN / Daily Mail (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Benchmark
Abstractive Text Summarization CNN / Daily Mail CoCoNet + CoCoPretrain ROUGE-1 44.50 # 11
ROUGE-2 21.55 # 7
ROUGE-L 41.24 # 16
Abstractive Text Summarization CNN / Daily Mail CoCoNet ROUGE-1 44.39 # 13
ROUGE-2 21.41 # 12
ROUGE-L 41.05 # 20

Methods


No methods listed for this paper. Add relevant methods here