Text Summarization

209 papers with code • 19 benchmarks • 51 datasets

Shortening a set of data computationally, to create a summary that represents the most important or relevant information within the original content (Source: Wikipedia).

Greatest papers with code

Big Bird: Transformers for Longer Sequences

tensorflow/models NeurIPS 2020

To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear.

Linguistic Acceptability Natural Language Inference +3

Attention Is All You Need

tensorflow/models NeurIPS 2017

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.

Ranked #2 on Multimodal Machine Translation on Multi30K (BLUE (DE-EN) metric)

Abstractive Text Summarization Constituency Parsing +2

A Neural Attention Model for Abstractive Sentence Summarization

tensorflow/models EMNLP 2015

Summarization based on text extraction is inherently limited, but generation-style abstractive methods have proven challenging to build.

Extractive Text Summarization Sentence Summarization

BARThez: a Skilled Pretrained French Sequence-to-Sequence Model

huggingface/transformers EMNLP 2021

We show BARThez to be very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT.

 Ranked #1 on Text Summarization on OrangeSum (using extra training data)

Natural Language Understanding Self-Supervised Learning +2

ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

huggingface/transformers 13 Jan 2020

This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism.

Ranked #5 on Text Summarization on GigaWord (using extra training data)

Abstractive Text Summarization Question Generation

PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

huggingface/transformers ICML 2020

Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summarization.

Abstractive Text Summarization

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

huggingface/transformers ACL 2020

We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.

Abstractive Text Summarization Denoising +5

Leveraging Pre-trained Checkpoints for Sequence Generation Tasks

huggingface/transformers TACL 2020

Unsupervised pre-training of large neural models has recently revolutionized Natural Language Processing.

Language understanding Machine Translation +5

SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders

google-research/google-research 2 Oct 2019

We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.

Abstractive Text Summarization Denoising

Better Fine-Tuning by Reducing Representational Collapse

pytorch/fairseq ICLR 2021

Although widely adopted, existing approaches for fine-tuning pre-trained language models have been shown to be unstable across hyper-parameter settings, motivating recent work on trust region methods.

Abstractive Text Summarization Cross-Lingual Natural Language Inference +1