Search Results for author:
Found 6 papers, 2 papers with code
Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic Evaluation
On the one hand, the human assessment of summarization quality conducted by linguistic experts is slow, expensive, and still not a standardized procedure.
Reliability of Human Evaluation for Text Summarization: Lessons Learned and Challenges Ahead
Based on our empirical analysis, we provide guidelines to ensure the reliability of expert and non-expert evaluations, and we determine the factors that might affect the reliability of the human evaluation.
Towards Hybrid Human-Machine Workflow for Natural Language Generation
In recent years, crowdsourcing has gained much attention from researchers to generate data for the Natural Language Generation (NLG) tools or to evaluate them.
Does Summary Evaluation Survive Translation to Other Languages?
The creation of a quality summarization dataset is an expensive, time-consuming effort, requiring the production and evaluation of summaries by both trained humans and machines.
Towards Human-Free Automatic Quality Evaluation of German Summarization
Evaluating large summarization corpora using humans has proven to be expensive from both the organizational and the financial perspective.
Towards a Reliable and Robust Methodology for Crowd-Based Subjective Quality Assessment of Query-Based Extractive Text Summarization
The intrinsic and extrinsic quality evaluation is an essential part of the summary evaluation methodology usually conducted in a traditional controlled laboratory environment.
