Linguistic Acceptability
47 papers with code • 5 benchmarks • 5 datasets
Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.
Image Source: Warstadt et al
Benchmarks
These leaderboards are used to track progress in Linguistic Acceptability
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Linguistic Acceptability models and implementations
GLUE
CoLA
ItaCoLA
RuCoLA
Latest papers
JCoLA: Japanese Corpus of Linguistic Acceptability
In this paper, we introduce JCoLA (Japanese Corpus of Linguistic Acceptability), which consists of 10, 020 sentences annotated with binary acceptability judgments.
NoCoLA: The Norwegian Corpus of Linguistic Acceptability
ltgoslo/nocola
•
•
While there has been a surge of large language models for Norwegian in recent years, we lack any tool to evaluate their understanding of grammaticality.
CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models
lijiazheng99/cue
•
•
We then generate text representations by perturbing the latent space which causes fluctuation in predictive uncertainty.
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
This paper proposes LM-CPPF, Contrastive Paraphrasing-guided Prompt-based Fine-tuning of Language Models, which leverages prompt-based few-shot paraphrasing using generative language models, especially large language models such as GPT-3 and OPT-175B, for data augmentation.
Revisiting Acceptability Judgements
We introduce CoLAC - Corpus of Linguistic Acceptability in Chinese, the first large-scale acceptability dataset for a non-Indo-European language.
Can BERT eat RuCoLA? Topological Data Analysis to Explain
Our results contribute to understanding the behavior of monolingual LMs in the acceptability classification task, provide insights into the functional roles of attention heads, and highlight the advantages of TDA-based approaches for analyzing LMs.
ScandEval: A Benchmark for Scandinavian Natural Language Processing
This paper introduces a Scandinavian benchmarking platform, ScandEval, which can benchmark any pretrained model on four different tasks in the Scandinavian languages.
ChatGPT: Jack of all trades, master of none
Our comparison of its results with available State-of-the-Art (SOTA) solutions showed that the average loss in quality of the ChatGPT model was about 25% for zero-shot and few-shot evaluation.
tasksource: A Dataset Harmonization Framework for Streamlined NLP Multi-Task Learning and Evaluation
We release a dataset annotation framework and dataset annotations for more than 500 English tasks\footnote{\url{https://github. com/sileod/tasksource}}.
RuCoLA: Russian Corpus of Linguistic Acceptability
Linguistic acceptability (LA) attracts the attention of the research community due to its many uses, such as testing the grammatical knowledge of language models and filtering implausible texts with acceptability classifiers.
