Browse > Natural Language Processing > Language Modelling

Language Modelling

588 papers with code · Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

( Image credit: Exploring the Limits of Language Modeling )

Leaderboards

Latest papers with code

BERTweet: A pre-trained language model for English Tweets

20 May 2020VinAIResearch/BERTweet

We present BERTweet, the first public large-scale pre-trained language model for English Tweets.

LANGUAGE MODELLING NAMED ENTITY RECOGNITION PART-OF-SPEECH TAGGING TEXT CLASSIFICATION

73
20 May 2020

How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?

17 May 2020bhattg/Decay-RNN-ACL-SRW2020

Long short-term memory (LSTM) networks and their variants are capable of encapsulating long-range dependencies, which is evident from their performance on a variety of linguistic tasks.

LANGUAGE MODELLING

1
17 May 2020

MicroNet for Efficient Language Modeling

16 May 2020mit-han-lab/neurips-micronet

In this paper, we provide the winning solution to the NeurIPS 2019 MicroNet Challenge in the language modeling track.

LANGUAGE MODELLING MODEL COMPRESSION NETWORK PRUNING QUANTIZATION

7
16 May 2020

AttViz: Online exploration of self-attention for transparent neural language modeling

12 May 2020SkBlaz/attviz

Neural language models are becoming the prevailing methodology for the tasks of query answering, text classification, disambiguation, completion and translation.

LANGUAGE MODELLING TEXT CLASSIFICATION

1
12 May 2020

Enabling Language Models to Fill in the Blanks

11 May 2020chrisdonahue/ilm

We show that this approach, which we call infilling by language modeling, can enable LMs to infill entire sentences effectively on three different domains: short stories, scientific abstracts, and lyrics.

LANGUAGE MODELLING TEXT INFILLING

14
11 May 2020

Token Manipulation Generative Adversarial Network for Text Generation

6 May 2020twidddj/tokmangan

MaskGAN opens the query for the conditional language model by filling in the blanks between the given tokens.

LANGUAGE MODELLING TEXT GENERATION

5
06 May 2020

Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction

4 May 2020raphael-sch/HC_Sentence_Summarization

Automatic sentence summarization produces a shorter version of a sentence, while preserving its most important information.

LANGUAGE MODELLING SEMANTIC SIMILARITY SEMANTIC TEXTUAL SIMILARITY UNSUPERVISED SENTENCE SUMMARIZATION

1
04 May 2020

A Simple Language Model for Task-Oriented Dialogue

2 May 2020budzianowski/multiwoz

Task-oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response.

DIALOGUE STATE TRACKING LANGUAGE MODELLING TRANSFER LEARNING

218
02 May 2020

UnifiedQA: Crossing Format Boundaries With a Single QA System

2 May 2020allenai/unifiedqa

As evidence, we use the latest advances in language modeling to build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats.

LANGUAGE MODELLING QUESTION ANSWERING

26
02 May 2020