About

Language modeling is the task of predicting the next word or character in a document.

( Image credit: Exploring the Limits of Language Modeling )

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Libraries

Subtasks

Datasets

Latest papers with code

When to Fold'em: How to answer Unanswerable questions

1 May 2021allenai/document-qa

We present 3 different question-answering models trained on the SQuAD2. 0 dataset -- BIDAF, DocumentQA and ALBERT Retro-Reader -- demonstrating the improvement of language models in the past three years.

LANGUAGE MODELLING QUESTION ANSWERING

386
01 May 2021

Hidden Backdoors in Human-Centric Language Models

1 May 2021lishaofeng/NLP_Backdoor

We are able to demonstrate the adversary's high success rate of attacks, while maintaining functionality for regular users, with triggers inconspicuous by the human administrators.

LANGUAGE MODELLING MACHINE TRANSLATION QUESTION ANSWERING

1
01 May 2021

XLM-T: A Multilingual Language Model Toolkit for Twitter

25 Apr 2021cardiffnlp/xlm-t

Language models are ubiquitous in current NLP, and their multilingual capacity has recently attracted considerable attention.

LANGUAGE MODELLING SENTIMENT ANALYSIS

45
25 Apr 2021

Improving Biomedical Pretrained Language Models with Knowledge

21 Apr 2021GanjinZero/KeBioLM

To this end, we propose KeBioLM, a biomedical pretrained language model that explicitly leverages knowledge from the UMLS knowledge bases.

ENTITY LINKING LANGUAGE MODELLING NAMED ENTITY RECOGNITION RELATION EXTRACTION

3
21 Apr 2021

Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?

21 Apr 2021timpal0l/ScandiSent

We demonstrate empirically that a large English language model coupled with modern machine translation outperforms native language models in most Scandinavian languages.

LANGUAGE MODELLING MACHINE TRANSLATION

3
21 Apr 2021

Differentiable Model Compression via Pseudo Quantization Noise

20 Apr 2021facebookresearch/diffq

We propose to add independent pseudo quantization noise to model parameters during training to approximate the effect of a quantization operator.

AUDIO SOURCE SEPARATION IMAGE CLASSIFICATION LANGUAGE MODELLING MODEL COMPRESSION QUANTIZATION

92
20 Apr 2021

Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model

20 Apr 2021ku-nlp/steganography-with-masked-lm

With advances in neural language models, the focus of linguistic steganography has shifted from edit-based approaches to generation-based ones.

LANGUAGE MODELLING

2
20 Apr 2021

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

19 Apr 2021gmpoli/electramed

The overwhelming amount of biomedical scientific texts calls for the development of effective language models able to tackle a wide range of biomedical natural language processing (NLP) tasks.

 Ranked #1 on Named Entity Recognition on BC5CDR (using extra training data)

DRUG–DRUG INTERACTION EXTRACTION LANGUAGE MODELLING MEDICAL NAMED ENTITY RECOGNITION QUESTION ANSWERING RELATION EXTRACTION

6
19 Apr 2021

Towards Open-World Text-Guided Face Image Generation and Manipulation

18 Apr 2021weihaox/TediGAN

To be specific, we propose a brand new paradigm of text-guided image generation and manipulation based on the superior characteristics of a pretrained GAN model.

LANGUAGE MODELLING SEMANTIC SEGMENTATION TEXT-TO-IMAGE GENERATION

103
18 Apr 2021

FedNLP: A Research Platform for Federated Learning in Natural Language Processing

18 Apr 2021FedML-AI/FedNLP

To facilitate FL research in NLP, we present the FedNLP, a research platform for federated learning in NLP.

FEDERATED LEARNING LANGUAGE MODELLING QUESTION ANSWERING TEXT CLASSIFICATION

99
18 Apr 2021