A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization

9 May 2018  ·  Li Wang, Junlin Yao, Yunzhe Tao, Li Zhong, Wei Liu, Qiang Du ·

In this paper, we propose a deep learning approach to tackle the automatic summarization tasks by incorporating topic information into the convolutional sequence-to-sequence (ConvS2S) model and using self-critical sequence training (SCST) for optimization. Through jointly attending to topics and word-level alignment, our approach can improve coherence, diversity, and informativeness of generated summaries via a biased probability generation mechanism. On the other hand, reinforcement training, like SCST, directly optimizes the proposed model with respect to the non-differentiable metric ROUGE, which also avoids the exposure bias during inference. We carry out the experimental evaluation with state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets. The empirical results demonstrate the superiority of our proposed method in the abstractive summarization.

PDF Abstract


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Text Summarization DUC 2004 Task 1 Reinforced-Topic-ConvS2S ROUGE-1 31.15 # 6
ROUGE-2 10.85 # 5
ROUGE-L 27.68 # 5
Text Summarization GigaWord Reinforced-Topic-ConvS2S ROUGE-1 36.92 # 25
ROUGE-2 18.29 # 23
ROUGE-L 34.58 # 23