Browse > Natural Language Processing > Language Modelling

Language Modelling

282 papers with code · Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

State-of-the-art leaderboards

Latest papers with code

Adversarial Dropout for Recurrent Neural Networks

22 Apr 2019sungraepark/adversarial_dropout_text_classification

Successful application processing sequential data, such as text and speech, requires an improved generalization performance of recurrent neural networks (RNNs).

LANGUAGE MODELLING TEXT CLASSIFICATION

0
22 Apr 2019

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

18 Apr 2019shelling203/SpecAugment

On LibriSpeech, we achieve 6. 8% WER on test-other without the use of a language model, and 5. 8% WER with shallow fusion with a language model.

END-TO-END SPEECH RECOGNITION LANGUAGE MODELLING SPEECH RECOGNITION

5
18 Apr 2019

Dynamic Evaluation of Transformer Language Models

17 Apr 2019benkrause/dynamiceval-transformer

This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation.

LANGUAGE MODELLING

12
17 Apr 2019

Sparseout: Controlling Sparsity in Deep Networks

17 Apr 2019najeebkhan/sparseout

We found that sparsity of the activations is favorable for language modelling performance while image classification benefits from denser activations.

IMAGE CLASSIFICATION LANGUAGE MODELLING

1
17 Apr 2019

Effective Estimation of Deep Generative Language Models

17 Apr 2019tom-pelsmaeker/deep-generative-lm

This paper contributes a sober view of the problem, a survey of techniques to address it, novel techniques, and extensions to the model.

LANGUAGE MODELLING

11
17 Apr 2019

Sameness Attracts, Novelty Disturbs, but Outliers Flourish in Fanfiction Online

16 Apr 2019yzjing/ao3

The nature of what people enjoy is not just a central question for the creative industry, it is a driving force of cultural evolution.

LANGUAGE MODELLING

2
16 Apr 2019

A Statistical Investigation of Long Memory in Language and Music

8 Apr 2019alecgt/RNN_long_memory

Representation and learning of long-range dependencies is a central challenge confronted in modern applications of machine learning to sequence data.

LANGUAGE MODELLING TIME SERIES

4
08 Apr 2019

Unsupervised Recurrent Neural Network Grammars

7 Apr 2019harvardnlp/urnng

On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese.

LANGUAGE MODELLING

46
07 Apr 2019

SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression

7 Apr 2019cbaziotis/seq3

The proposed model does not require parallel text-summary pairs, achieving promising results in unsupervised sentence compression on benchmark datasets.

LANGUAGE MODELLING UNSUPERVISED SENTENCE COMPRESSION

6
07 Apr 2019

Jasper: An End-to-End Convolutional Neural Acoustic Model

5 Apr 2019NVIDIA/OpenSeq2Seq

In this paper, we report state-of-the-art results on LibriSpeech among end-to-end speech recognition models without any external training data.

END-TO-END SPEECH RECOGNITION LANGUAGE MODELLING SPEECH RECOGNITION

755
05 Apr 2019