Linguistic Acceptability
47 papers with code • 5 benchmarks • 5 datasets
Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.
Image Source: Warstadt et al
Benchmarks
These leaderboards are used to track progress in Linguistic Acceptability
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Linguistic Acceptability models and implementations
GLUE
CoLA
ItaCoLA
RuCoLA
Most implemented papers
TinyBERT: Distilling BERT for Natural Language Understanding
huawei-noah/Pretrained-Language-Model
•
•
Findings of the Association for Computational Linguistics 2020
To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models.
SpanBERT: Improving Pre-training by Representing and Predicting Spans
facebookresearch/SpanBERT
•
•
TACL 2020
We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text.
Masked Language Model Scoring
awslabs/mlm-scoring
•
•
ACL 2020
Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are computed by masking tokens one by one.
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model.
Q8BERT: Quantized 8Bit BERT
NervanaSystems/nlp-architect
•
•
Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks.
RealFormer: Transformer Likes Residual Attention
google-research/google-research
•
•
Findings (ACL) 2021
Transformer is the backbone of modern NLP models.
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
huggingface/transformers
•
•
EMNLP (sustainlp) 2020
Humans read and write hundreds of billions of messages every day.
How to Train BERT with an Academic Budget
peteriz/academic-budget-bert
•
•
EMNLP 2021
While large language models a la BERT are used ubiquitously in NLP, pretraining them is considered a luxury that only a few well-funded industry labs can afford.
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing.
GeDi: Generative Discriminator Guided Sequence Generation
salesforce/GeDi
•
•
Findings (EMNLP) 2021
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate.
