Search Results for author:
Found 9 papers, 2 papers with code
Results of the WMT20 Metrics Shared Task
Participants were asked to score the outputs of the translation systems competing in the WMT20 News Translation Task with automatic metrics.
Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain
Contrary to previous years’ editions, this year we acquired our own human ratings based on expert-based human evaluation via Multidimensional Quality Metrics (MQM).
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics
Automatic metrics are fundamental for the development and evaluation of machine translation systems.
Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation
Accurate, automatic evaluation of machine translation is critical for system tuning, and evaluating progress in the field.
Towards Efficient Machine Translation Evaluation by Modelling Annotators
In this paper we show that the quality control mechanism is overly conservative, which increases the time and expense of the evaluation.
Sequence Effects in Crowdsourced Annotations
Manual data annotation is a vital component of NLP research.
