Coherence Evaluation
13 papers with code • 2 benchmarks • 1 datasets
Evaluating the overall coherence of text as measured by its readability and flow through ideas.
Benchmarks
These leaderboards are used to track progress in Coherence Evaluation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
GCDC
Most implemented papers
Transformer Models for Text Coherence Assessment
tushar117/Transformer-Models-for-Text-Coherence-Assessment
•
•
Coherence is an important aspect of text quality and is crucial for ensuring its readability.
Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
To date there has been very little work on assessing discourse coherence methods on real-world data.
Neural RST-based Evaluation of Discourse Coherence
grig-guz/coherence-rst
•
•
Asian Chapter of the Association for Computational Linguistics 2020
We evaluate our approach on the Grammarly Corpus for Discourse Coherence (GCDC) and show that when ensembled with the current state of the art, we can achieve the new state of the art accuracy on this benchmark.
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models
AnneBeyer/coherencegym
•
•
NAACL 2021
Coherent discourse is distinguished from a mere collection of utterances by the satisfaction of a diverse set of constraints, for example choice of expression, logical relation between denoted events, and implicit compatibility with world-knowledge.
Towards Quantifiable Dialogue Coherence Evaluation
James-Yip/QuantiDCE
•
•
ACL 2021
To address these limitations, we propose Quantifiable Dialogue Coherence Evaluation (QuantiDCE), a novel framework aiming to train a quantifiable dialogue coherence metric that can reflect the actual human rating standards.
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations
We also show that DEAM can distinguish between coherent and incoherent dialogues generated by baseline manipulations, whereas those baseline models cannot detect incoherent examples generated by DEAM.
SNaC: Coherence Error Detection for Narrative Summarization
tagoyal/snac
•
•
In this work, we introduce SNaC, a narrative coherence evaluation framework rooted in fine-grained annotations for long summaries.
GisPy: A Tool for Measuring Gist Inference Score in Text
Decision making theories such as Fuzzy-Trace Theory (FTT) suggest that individuals tend to rely on gist, or bottom-line meaning, in the text when making decisions.
Transform, Contrast and Tell: Coherent Entity-Aware Multi-Image Captioning
Experiments also show that the generated captions are more coherent than that of baselines according to caption entity scores, caption Rouge scores, the two proposed coherence evaluation metrics, and human evaluations.
The Problem of Coherence in Natural Language Explanations of Recommendations
jmraczynski/cer
•
•
Providing natural language explanations for recommendations is particularly useful from the perspective of a non-expert user.
