Search Results for author:
Found 10 papers, 3 papers with code
What happens if you treat ordinal ratings as interval data? Human evaluations in NLP are even more under-powered than you think
Previous work has shown that human evaluations in NLP are notoriously under-powered.
Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions
Human assessment remains the most trusted form of evaluation in NLG, but highly diverse approaches and a proliferation of different quality criteria used by researchers make it difficult to compare results and draw conclusions across papers, with adverse implications for meta-evaluation and reproducibility.
Disentangling the Properties of Human Evaluation Methods: A Classification System to Support Comparability, Meta-Evaluation and Reproducibility Testing
Current standards for designing and reporting human evaluations in NLP mean it is generally unclear which evaluations are comparable and can be expected to yield similar results when applied to the same system outputs.
Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices
Automatic metrics are extensively used to evaluate natural language processing systems.
OTTers: One-turn Topic Transitions for Open-Domain Dialogue
Mixed initiative in open-domain dialogue requires a system to pro-actively introduce new topics.
Semantic Noise Matters for Neural Natural Language Generation
Neural natural language generation (NNLG) systems are known for their pathological outputs, i. e. generating text which is unrelated to the input specification.
Ranked #3 on
Data-to-Text Generation
on Cleaned E2E NLG Challenge
Toward Bayesian Synchronous Tree Substitution Grammars for Sentence Planning
Developing conventional natural language generation systems requires extensive attention from human experts in order to craft complex sets of sentence planning rules.
Psycholinguistic Models of Sentence Processing Improve Sentence Readability Ranking
While previous research on readability has typically focused on document-level measures, recent work in areas such as natural language generation has pointed out the need of sentence-level readability measures.
From OpenCCG to AI Planning: Detecting Infeasible Edges in Sentence Generation
The search space in grammar-based natural language generation tasks can get very large, which is particularly problematic when generating long utterances or paragraphs.
