Browse > Natural Language Processing > Language Modelling

Language Modelling

588 papers with code ยท Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

( Image credit: Exploring the Limits of Language Modeling )

Leaderboards

Latest papers without code

ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition

21 May 2020

In this paper we present state-of-the-art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self-attentive simple recurrent unit (SRU) for language modeling.

DATA AUGMENTATION LANGUAGE MODELLING SPEECH RECOGNITION

Investigation of Large-Margin Softmax in Neural Language Modeling

20 May 2020

After that, we apply the best norm-scaling setup in combination with various margins and conduct neural language models rescoring experiments in automatic speech recognition.

FACE RECOGNITION LANGUAGE MODELLING SPEECH RECOGNITION

Early Stage LM Integration Using Local and Global Log-Linear Combination

20 May 2020

This is compared to a global renormalization scheme which is equivalent to applying shallow fusion in training.

LANGUAGE MODELLING SPEECH RECOGNITION

Iterative Pseudo-Labeling for Speech Recognition

19 May 2020

In particular, IPL fine-tunes an existing model at each iteration using both labeled data and a subset of unlabeled data.

DATA AUGMENTATION LANGUAGE MODELLING SPEECH RECOGNITION

Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text

19 May 2020

Here, we propose a conceptually simple method for training instruction-following agents with deep RL that are robust to natural human instructions.

LANGUAGE MODELLING REPRESENTATION LEARNING TRANSFER LEARNING

Table Search Using a Deep Contextualized Language Model

19 May 2020

Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks.

LANGUAGE MODELLING

Improving Proper Noun Recognition in End-to-End ASR By Customization of the MWER Loss Criterion

19 May 2020

Proper nouns present a challenge for end-to-end (E2E) automatic speech recognition (ASR) systems in that a particular name may appear only rarely during training, and may have a pronunciation similar to that of a more common word.

END-TO-END SPEECH RECOGNITION LANGUAGE MODELLING SPEECH RECOGNITION

Approaches to Improving Recognition of Underrepresented Named Entities in Hybrid ASR Systems

18 May 2020

In this paper, we present a series of complementary approaches to improve the recognition of underrepresented named entities (NE) in hybrid ASR systems without compromising overall word error rate performance.

LANGUAGE MODELLING

The NTNU System at the Interspeech 2020 Non-Native Children's Speech ASR Challenge

18 May 2020

This paper describes the NTNU ASR system participating in the Interspeech 2020 Non-Native Children's Speech ASR Challenge supported by the SIG-CHILD group of ISCA.

DATA AUGMENTATION LANGUAGE MODELLING