Language modeling is the task of predicting the next word or character in a document.
( Image credit: Exploring the Limits of Language Modeling )
|TREND||DATASET||BEST METHOD||PAPER TITLE||PAPER||CODE||COMPARE|
We propose a new benchmark corpus to be used for measuring progress in statistical language modeling.
#14 best model for Language Modelling on One Billion Word
These approaches corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens.
#3 best model for Sentiment Analysis on SST-2 Binary classification
Language models have become a key step to achieve state-of-the art results in many different Natural Language Processing (NLP) tasks.
We measure the performance of CamemBERT compared to multilingual models in multiple downstream tasks, namely part-of-speech tagging, dependency parsing, named-entity recognition, and natural language inference.
We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale.
As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging.
#5 best model for Semantic Textual Similarity on MRPC
Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text.
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
SOTA for Reading Comprehension on RACE