About

Text generation is the task of generating text with the goal of appearing indistinguishable to human-written text.

( Image credit: Adversarial Ranking for Language Generation )

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Subtasks

Datasets

Greatest papers with code

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

NeurIPS 2020 huggingface/transformers

Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks.

QUESTION ANSWERING TEXT GENERATION

Plug and Play Language Models: A Simple Approach to Controlled Text Generation

ICLR 2020 huggingface/transformers

Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities.

LANGUAGE MODELLING TEXT GENERATION

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

ACL 2020 huggingface/transformers

We evaluate a number of noising approaches, finding the best performance by both randomly shuffling the order of the original sentences and using a novel in-filling scheme, where spans of text are replaced with a single mask token.

ABSTRACTIVE TEXT SUMMARIZATION DENOISING MACHINE TRANSLATION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING TEXT GENERATION

HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 2019huggingface/transformers

Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks.

TEXT GENERATION TRANSFER LEARNING

Language Models are Unsupervised Multitask Learners

Preprint 2019 huggingface/transformers

Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.

 Ranked #1 on Language Modelling on enwik8 (using extra training data)

4 COMMON SENSE REASONING DATA-TO-TEXT GENERATION DOCUMENT SUMMARIZATION LANGUAGE MODELLING MACHINE TRANSLATION MULTI-TASK LEARNING QUESTION ANSWERING READING COMPREHENSION

Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach

20 Feb 2021google-research/google-research

An ideal environment for evaluating dialog systems, also known as the Turing test, needs to involve human interaction, which is usually not affordable for large-scale experiments.

TEXT GENERATION

Stepwise Extractive Summarization and Planning with Structured Transformers

EMNLP 2020 google-research/google-research

We propose encoder-centric stepwise models for extractive summarization using structured transformers -- HiBERT and Extended Transformers.

TABLE-TO-TEXT GENERATION

fairseq: A Fast, Extensible Toolkit for Sequence Modeling

NAACL 2019 facebookresearch/fairseq-py

fairseq is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks.

LANGUAGE MODELLING TEXT GENERATION

Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

7 Oct 2016facebookresearch/fairseq-py

We observe that our method consistently outperforms BS and previously proposed techniques for diverse decoding from neural sequence models.

IMAGE CAPTIONING MACHINE TRANSLATION QUESTION GENERATION TEXT GENERATION TIME SERIES