Search Results for author: David M. Howcroft

Found 9 papers, 2 papers with code

Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions

no code implementations INLG (ACL) 2020 David M. Howcroft, Anya Belz, Miruna-Adriana Clinciu, Dimitra Gkatzia, Sadid A. Hasan, Saad Mahamood, Simon Mille, Emiel van Miltenburg, Sashank Santhanam, Verena Rieser

Human assessment remains the most trusted form of evaluation in NLG, but highly diverse approaches and a proliferation of different quality criteria used by researchers make it difficult to compare results and draw conclusions across papers, with adverse implications for meta-evaluation and reproducibility.

Experimental Design

Disentangling the Properties of Human Evaluation Methods: A Classification System to Support Comparability, Meta-Evaluation and Reproducibility Testing

no code implementations INLG (ACL) 2020 Anya Belz, Simon Mille, David M. Howcroft

Current standards for designing and reporting human evaluations in NLP mean it is generally unclear which evaluations are comparable and can be expected to yield similar results when applied to the same system outputs.

Semantic Noise Matters for Neural Natural Language Generation

1 code implementation WS 2019 Ondřej Dušek, David M. Howcroft, Verena Rieser

Neural natural language generation (NNLG) systems are known for their pathological outputs, i. e. generating text which is unrelated to the input specification.

Data-to-Text Generation

Toward Bayesian Synchronous Tree Substitution Grammars for Sentence Planning

no code implementations WS 2018 David M. Howcroft, Dietrich Klakow, Vera Demberg

Developing conventional natural language generation systems requires extensive attention from human experts in order to craft complex sets of sentence planning rules.

Text Generation

Psycholinguistic Models of Sentence Processing Improve Sentence Readability Ranking

no code implementations EACL 2017 David M. Howcroft, Vera Demberg

While previous research on readability has typically focused on document-level measures, recent work in areas such as natural language generation has pointed out the need of sentence-level readability measures.

Information Retrieval Text Generation +1

From OpenCCG to AI Planning: Detecting Infeasible Edges in Sentence Generation

no code implementations COLING 2016 Maximilian Schwenger, {\'A}lvaro Torralba, Joerg Hoffmann, David M. Howcroft, Vera Demberg

The search space in grammar-based natural language generation tasks can get very large, which is particularly problematic when generating long utterances or paragraphs.

Text Generation

Cannot find the paper you are looking for? You can Submit a new open access paper.