Abstractive Text Summarization

154 papers with code • 7 benchmarks • 32 datasets

Abstractive Text Summarization is the task of generating a short and concise summary that captures the salient ideas of the source text. The generated summaries potentially contain new phrases and sentences that may not appear in the source text.

Source: Generative Adversarial Network for Abstractive Text Summarization

Image credit: Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond

Greatest papers with code

Attention Is All You Need

tensorflow/models NeurIPS 2017

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.

Abstractive Text Summarization Constituency Parsing +1

ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

huggingface/transformers 13 Jan 2020

This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism.

Ranked #4 on Question Generation on SQuAD1.1 (using extra training data)

Abstractive Text Summarization Question Generation

PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

huggingface/transformers ICML 2020

Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summarization.

Abstractive Text Summarization

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

huggingface/transformers ACL 2020

We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.

Abstractive Text Summarization Denoising +4

SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders

google-research/google-research 2 Oct 2019

We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.

Abstractive Text Summarization Denoising

Better Fine-Tuning by Reducing Representational Collapse

pytorch/fairseq ICLR 2021

Although widely adopted, existing approaches for fine-tuning pre-trained language models have been shown to be unstable across hyper-parameter settings, motivating recent work on trust region methods.

Abstractive Text Summarization Cross-Lingual Natural Language Inference

Pre-trained Language Model Representations for Language Generation

pytorch/fairseq NAACL 2019

Pre-trained language model representations have been successful in a wide range of language understanding tasks.

Abstractive Text Summarization Language Modelling +2

Pay Less Attention with Lightweight and Dynamic Convolutions

pytorch/fairseq ICLR 2019

We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements.

Abstractive Text Summarization Language Modelling +1

Classical Structured Prediction Losses for Sequence to Sequence Learning

pytorch/fairseq NAACL 2018

There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam.

Abstractive Text Summarization Machine Translation +1

Sample Efficient Text Summarization Using a Single Pre-Trained Transformer

tensorflow/tensor2tensor 21 May 2019

Language model (LM) pre-training has resulted in impressive performance and sample efficiency on a variety of language understanding tasks.

Abstractive Text Summarization Language Modelling