Browse > Natural Language Processing > Language Modelling

Language Modelling

280 papers with code · Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

State-of-the-art leaderboards

Greatest papers with code

Exploring the Limits of Language Modeling

7 Feb 2016tensorflow/models

In this work we explore recent advances in Recurrent Neural Networks for large scale Language Modeling, a task central to language understanding.

LANGUAGE MODELLING

Semi-supervised Sequence Learning

NeurIPS 2015 tensorflow/models

In our experiments, we find that long short term memory recurrent networks after being pretrained with the two approaches are more stable and generalize better.

LANGUAGE MODELLING TEXT CLASSIFICATION

One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling

11 Dec 2013tensorflow/models

We propose a new benchmark corpus to be used for measuring progress in statistical language modeling.

LANGUAGE MODELLING

Universal Transformers

ICLR 2019 tensorflow/tensor2tensor

Feed-forward and convolutional architectures have recently been shown to achieve superior results on some sequence modeling tasks such as machine translation, with the added advantage that they concurrently process all inputs in the sequence, leading to easy parallelization and faster training times.

LANGUAGE MODELLING LEARNING TO EXECUTE MACHINE TRANSLATION

Discrete Autoencoders for Sequence Models

ICLR 2018 tensorflow/tensor2tensor

We propose to improve the representation in sequence models by augmenting current approaches with an autoencoder that is forced to compress the sequence through an intermediate discrete latent space.

LANGUAGE MODELLING MACHINE TRANSLATION

Neural Network Language Modeling with Letter-based Features and Importance Sampling

ICASSP 2018 kaldi-asr/kaldi

In this paper we describe an extension of the Kaldi software toolkit to support neural-based language modeling, intended for use in automatic speech recognition (ASR) and related tasks.

LANGUAGE MODELLING SPEECH RECOGNITION

Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baseline

27 Mar 2018kaldi-asr/kaldi

This paper describes a new baseline system for automatic speech recognition (ASR) in the CHiME-4 challenge to promote the development of noisy ASR in speech processing communities by providing 1) state-of-the-art system with a simplified single system comparable to the complicated top systems in the challenge, 2) publicly available and reproducible recipe through the main repository in the Kaldi speech recognition toolkit.

DISTANT SPEECH RECOGNITION LANGUAGE MODELLING NOISY SPEECH RECOGNITION SPEECH ENHANCEMENT

Purely sequence-trained neural networks for ASR based on lattice-free MMI

INTERSPEECH 2016 2016 kaldi-asr/kaldi

Models trained with LFMMI provide a relative word error rate reduction of ∼11. 5%, over those trained with cross-entropy objective function, and ∼8%, over those trained with cross-entropy and sMBR objective functions.

LANGUAGE MODELLING LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION SPEECH RECOGNITION

Deep contextualized word representations

HLT 2018 zalandoresearch/flair

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e. g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i. e., to model polysemy).

CITATION INTENT CLASSIFICATION COREFERENCE RESOLUTION LANGUAGE MODELLING NAMED ENTITY RECOGNITION (NER) NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC ROLE LABELING SENTIMENT ANALYSIS