About

Language modeling is the task of predicting the next word or character in a document.

( Image credit: Exploring the Limits of Language Modeling )

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Libraries

Subtasks

Datasets

Latest papers without code

Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning

4 May 2021

Transformer-based language model approaches to automated story generation currently provide state-of-the-art results.

LANGUAGE MODELLING

HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish

4 May 2021

Therefore, this paper presents the first ablation study focused on Polish, which, unlike the isolating English language, is a fusional language.

LANGUAGE MODELLING

Impact of Gender Debiased Word Embeddings in Language Modeling

3 May 2021

Gender, race and social biases have recently been detected as evident examples of unfairness in applications of Natural Language Processing.

FAIRNESS LANGUAGE MODELLING WORD EMBEDDINGS

On the limit of English conversational speech recognition

3 May 2021

Compensation of the decoder model with the probability ratio approach allows more efficient integration of an external language model, and we report 5. 9% and 11. 5% WER on the SWB and CHM parts of Hub5'00 with very simple LSTM models.

ENGLISH CONVERSATIONAL SPEECH RECOGNITION LANGUAGE MODELLING

Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review

3 May 2021

We indeed find that the pre-trained BERT model reduces review volume by 30% in TAR workflows simulated on the RCV1-v2 newswire collection.

ACTIVE LEARNING LANGUAGE MODELLING TEXT CLASSIFICATION

Unsupervised Document Expansion for Information Retrieval with Stochastic Text Generation

3 May 2021

In this paper, we propose an Unsupervised Document Expansion with Generation (UDEG) framework with a pre-trained language model, which generates diverse supplementary sentences for the original document without using labels on query-document pairs for training.

INFORMATION RETRIEVAL LANGUAGE MODELLING TEXT GENERATION

Larger-Scale Transformers for Multilingual Masked Language Modeling

2 May 2021

Our model also outperforms the RoBERTa-Large model on several English tasks of the GLUE benchmark by 0. 3% on average while handling 99 more languages.

LANGUAGE MODELLING

An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets

30 Apr 2021

The evaluated baseline precision of coreference relation extraction on the corpus is 71, that is higher the results reached on other Russian corpora.

LANGUAGE MODELLING RELATION EXTRACTION

Evaluating Groundedness in Dialogue Systems: The BEGIN Benchmark

30 Apr 2021

To facilitate evaluation of such metrics, we introduce the Benchmark for Evaluation of Grounded INteraction (BEGIN).

LANGUAGE MODELLING NATURAL LANGUAGE INFERENCE

The Interspeech Zero Resource Speech Challenge 2021: Spoken language modelling

29 Apr 2021

We present the Zero Resource Speech Challenge 2021, which asks participants to learn a language model directly from audio, without any text or labels.

LANGUAGE MODELLING