Search Results for author:
Found 12 papers, 0 papers with code
Opening up Minds with Argumentative Dialogues
We show that while both models perform closely in terms of opening up minds, the argument-based model is significantly better on other dialogue properties such as engagement and clarity.
Identifying Annotator Bias: A new IRT-based method for bias identification
Our interpretation of IRT offers an original bias identification method that can be used to compare annotators{'} bias and characterise annotation disagreement.
Agreement is overrated: A plea for correlation to assess human evaluation reliability
Following Sampson and Babarczy (2008), Lommel et al. (2014), Joshi et al. (2016) and Amidei et al. (2018b), such phenomena can be explained in terms of irreducible human language variability.
The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations
Rating and Likert scales are widely used in evaluation experiments to measure the quality of Natural Language Generation (NLG) systems.
Evaluation methodologies in Automatic Question Generation 2013-2018
In the last few years Automatic Question Generation (AQG) has attracted increasing interest.
Rethinking the Agreement in Human Evaluation Tasks
For this reason, we believe that annotation schemes for natural language generation tasks that are aimed at evaluating language quality need to be treated with great care.
A syllogistic system for propositions with intermediate quantifiers
This paper describes a formalism that subsumes Peterson's intermediate quantifier syllogistic system, and extends the ideas by van Eijck on Aristotle's logic.
A model of suspense for narrative generation
Most work on automatic generation of narratives, and more specifically suspenseful narrative, has focused on detailed domain-specific modelling of character psychology and plot structure.
Measuring Non-cooperation in Dialogue
In contrast, we start from rules for normal/correct dialogue behaviour - i. e., a dialogue game - which in principle can be derived from a corpus of cooperative dialogues, and provide a quantitative measure for the degree to which participants comply with these rules.
