Browse > Natural Language Processing > Language Modelling

Language Modelling

360 papers with code · Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

State-of-the-art leaderboards

Latest papers with code

LXMERT: Learning Cross-Modality Encoder Representations from Transformers

20 Aug 2019airsplay/lxmert

In LXMERT, we build a large-scale Transformer model that consists of three encoders: an object relationship encoder, a language encoder, and a cross-modality encoder.

LANGUAGE MODELLING QUESTION ANSWERING VISUAL QUESTION ANSWERING VISUAL REASONING

71
20 Aug 2019

Universal Adversarial Triggers for NLP

20 Aug 2019Eric-Wallace/universal-triggers

We define universal adversarial triggers: input-agnostic sequences of tokens that trigger a model to produce a specific prediction when concatenated to any input from a dataset.

LANGUAGE MODELLING READING COMPREHENSION

2
20 Aug 2019

VisualBERT: A Simple and Performant Baseline for Vision and Language

9 Aug 2019uclanlp/visualbert

We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.

LANGUAGE MODELLING VISUAL QUESTION ANSWERING VISUAL REASONING

36
09 Aug 2019

On the Variance of the Adaptive Learning Rate and Beyond

8 Aug 2019LiyuanLucasLiu/RAdam

The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.

IMAGE CLASSIFICATION LANGUAGE MODELLING MACHINE TRANSLATION STOCHASTIC OPTIMIZATION

1,090
08 Aug 2019

hULMonA: The Universal Language Model in Arabic

WS 2019 aub-mind/hULMonA

Experiment results show that the developed hULMonA and multi-lingual ULM are able to generalize well to multiple Arabic data sets and achieve new state of the art results in Arabic Sentiment Analysis for some of the tested sets.

ARABIC SENTIMENT ANALYSIS LANGUAGE MODELLING TEXT CLASSIFICATION TRANSFER LEARNING

6
01 Aug 2019

What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models

31 Jul 2019aetting/lm-diagnostics

Pre-training by language modeling has become a popular and successful approach to NLP tasks, but we have yet to understand exactly what linguistic capacities these pre-training processes confer upon models.

LANGUAGE MODELLING

12
31 Jul 2019

Simple Unsupervised Summarization by Contextual Matching

ACL 2019 jzhou316/Unsupervised-Sentence-Summarization

We propose an unsupervised method for sentence summarization using only language modeling.

LANGUAGE MODELLING

3
31 Jul 2019

A Hybrid Neural Network Model for Commonsense Reasoning

27 Jul 2019namisan/mt-dnn

An HNN consists of two component models, a masked language model and a semantic similarity model, which share a BERT-based contextual encoder but use different model-specific input and output layers.

LANGUAGE MODELLING SEMANTIC TEXTUAL SIMILARITY

946
27 Jul 2019

Language Modelling for Sound Event Detection with Teacher Forcing and Scheduled Sampling

19 Jul 2019dr-costas/SEDLM

On the contrary, with our method there is a decrease of 10% at F1 score and an increase of 11% at ER for the TUT-SED Synthetic 2016 dataset.

LANGUAGE MODELLING MACHINE TRANSLATION SOUND EVENT DETECTION

3
19 Jul 2019