Linguistic Acceptability
44 papers with code • 5 benchmarks • 5 datasets
Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.
Image Source: Warstadt et al
Benchmarks
These leaderboards are used to track progress in Linguistic Acceptability
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Linguistic Acceptability models and implementations
GLUE
CoLA
ItaCoLA
RuCoLA
Most implemented papers
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
google-research/bert
•
•
NAACL 2019
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
pytorch/fairseq
•
•
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
google-research/ALBERT
•
•
ICLR 2020
Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
huggingface/transformers
•
•
arXiv 2019
Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
huggingface/transformers
•
•
NeurIPS 2019
As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging.
FNet: Mixing Tokens with Fourier Transforms
google-research/google-research
•
•
NAACL 2022
At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing the fastest models across all sequence lengths on GPUs (and across relatively shorter lengths on TPUs).
Big Bird: Transformers for Longer Sequences
google-research/bigbird
•
•
NeurIPS 2020
To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear.
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
microsoft/DeBERTa
•
•
ICLR 2021
Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
Multi-Task Deep Neural Networks for Natural Language Understanding
In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks.
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
pytorch/fairseq
•
•
Preprint 2022
While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind.
