Search Results for author:
Found 42 papers, 6 papers with code
Microsoft Speech Language Translation (MSLT) Corpus: The IWSLT 2016 release for English, French and German
We describe the Microsoft Speech Language Translation (MSLT) corpus, which was created in order to evaluate end-to-end conversational speech translation quality.
Findings of the IWSLT 2022 Evaluation Campaign
The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation.
Overview of the IWSLT 2017 Evaluation Campaign
The IWSLT 2017 evaluation campaign has organised three tasks.
Domain Adaptation of Document-Level NMT in IWSLT19
We describe our four NMT systems submitted to the IWSLT19 shared task in English→Czech text-to-text translation of TED talks.
Findings of the 2021 Conference on Machine Translation (WMT21)
This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.
Machine Translation Human Evaluation: an investigation of evaluation based on Post-Editing and its relation with Direct Assessment
In this paper we present an analysis of the two most prominent methodologies used for the human evaluation of MT quality, namely evaluation based on Post-Editing (PE) and evaluation based on Direct Assessment (DA).
Preliminary WMT24 Ranking of General MT Systems and LLMs
This is the preliminary ranking of WMT24 General MT systems based on automatic metrics.
Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies
Ten years ago a single metric, BLEU, governed progress in machine translation research.
GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4
This paper introduces GEMBA-MQM, a GPT-based evaluation metric designed to detect translation quality errors, specifically for the quality estimation setting without the need for human reference translations.
Large Language Models Are State-of-the-Art Evaluators of Translation Quality
We describe GEMBA, a GPT-based metric for assessment of translation quality, which works both with a reference translation and without.
Searching for a higher power in the human evaluation of MT
In MT evaluation, pairwise comparisons are conducted to identify the better system.
The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
This paper presents the JHU-Microsoft joint submission for WMT 2021 quality estimation shared task.
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation
Automatic metrics are commonly used as the exclusive tool for declaring the superiority of one machine translation system's quality over another.
On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs
Recent studies emphasize the need of document context in human evaluation of machine translations, but little research has been done on the impact of user interfaces on annotator productivity and the reliability of assessments.
Findings of the 2020 Conference on Machine Translation (WMT20)
In the news task, participants were asked to build machine translation systems for any of 11 language pairs, to be evaluated on test sets consisting mainly of news stories.
Assessing Human-Parity in Machine Translation on the Segment Level
Recent machine translation shared tasks have shown top-performing systems to tie or in some cases even outperform human translation.
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN
The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation.
Multilingual Whispers: Generating Paraphrases with Translation
Naturally occurring paraphrase data, such as multiple news stories about the same event, is a useful but rare resource.
Findings of the 2019 Conference on Machine Translation (WMT19)
This paper presents the results of the premier shared task organized alongside the Conference on Machine Translation (WMT) 2019.
Findings of the WMT 2019 Shared Task on Automatic Post-Editing
Seven teams participated in the English-German task, with a total of 18 submitted runs.
Findings of the WMT 2019 Shared Tasks on Quality Estimation
We report the results of the WMT19 shared task on Quality Estimation, i. e. the task of predicting the quality of the output of machine translation systems given just the source text and the hypothesis translations.
Findings of the 2018 Conference on Machine Translation (WMT18)
This paper presents the results of the premier shared task organized alongside the Conference on Machine Translation (WMT) 2018.
Appraise Evaluation Framework for Machine Translation
We present Appraise, an open-source framework for crowd-based annotation tasks, notably for evaluation of machine translation output.
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
Involving Language Professionals in the Evaluation of Machine Translation
Significant breakthroughs in machine translation only seem possible if human translators are taken into the loop.
META-SHARE v2: An Open Network of Repositories for Language Resources including Data and Tools
We explain the underlying motivation for such a distributed repository for metadata storage and give a detailed overview on the META-SHARE application and its various components.
The ML4HMT Workshop on Optimising the Division of Labour in Hybrid Machine Translation
We describe the Shared Task on Applying Machine Learning Techniques to Optimise the Division of Labour in Hybrid Machine Translation (ML4HMT) which aims to foster research on improved system combination approaches for machine translation (MT).
A Richly Annotated, Multilingual Parallel Corpus for Hybrid Machine Translation
This corpus aims to serve as a basic resource for further research on whether hybrid machine translation algorithms and system combination techniques can benefit from additional (linguistically motivated, decoding, and runtime) information provided by the different systems involved.
