Search Results for author:
Found 25 papers, 3 papers with code
On the Same Page? Comparing Inter-Annotator Agreement in Sentence and Document Level Human Machine Translation Evaluation
Document-level evaluation of machine translation has raised interest in the community especially since responses to the claims of “human parity” (Toral et al., 2018; Läubli et al., 2018) with document-level human evaluations have been published.
MT-Pese: Machine Translation and Post-Editese
This paper introduces the MT-Pese project, which aims at researching the post-editese phenomena in machine translated texts.
TransCasm: A Bilingual Corpus of Sarcastic Tweets
Sarcasm is extensively used in User Generated Content (UGC) in order to express one’s discontent, especially through blogs, forums, or social media such as Twitter.
Achievements of the PRINCIPLE Project: Promoting MT for Croatian, Icelandic, Irish and Norwegian
This paper provides an overview of the main achievements of the completed PRINCIPLE project, a 2-year action funded by the European Commission under the Connecting Europe Facility (CEF) programme.
DELA Project: Document-level Machine Translation Evaluation
This paper presents the results of the DELA Project.
How Much Context Span is Enough? Examining Context-Related Issues for Document-level MT
This paper analyses how much context span is necessary to solve different context-related issues, namely, reference, ellipsis, gender, number, lexical ambiguity, and terminology when translating from English into Portuguese.
Towards Document-Level Human MT Evaluation: On the Issues of Annotator Agreement, Effort and Misevaluation
Document-level human evaluation of machine translation (MT) has been raising interest in the community.
DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues
Recently, the Machine Translation (MT) community has become more interested in document-level evaluation especially in light of reactions to claims of “human parity”, since examining the quality at the level of the document rather than at the sentence level allows for the assessment of suprasentential context, providing a more reliable evaluation.
A human evaluation of English-Irish statistical and neural machine translation
With official status in both Ireland and the EU, there is a need for high-quality English-Irish (EN-GA) machine translation (MT) systems which are suitable for use in a professional translation environment.
Document-Level Machine Translation Evaluation Project: Methodology, Effort and Inter-Annotator Agreement
Document-level (doc-level) human eval-uation of machine translation (MT) has raised interest in the community after a fewattempts have disproved claims of “human parity” (Toral et al., 2018; Laubli et al., 2018).
On Context Span Needed for Machine Translation Evaluation
Despite increasing efforts to improve evaluation of machine translation (MT) by going beyond the sentence level to the document level, the definition of what exactly constitutes a {``}document level{''} is still not clear.
A Set of Recommendations for Assessing Human-Machine Parity in Language Translation
The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations.
What Influences the Features of Post-editese? A Preliminary Study
While a number of studies have shown evidence of translationese phenomena, that is, statistical differences between original texts and translated texts (Gellerstam, 1986), results of studies searching for translationese features in postedited texts (what has been called {''}posteditese{''} (Daems et al., 2017)) have presented mixed results.
Are ambiguous conjunctions problematic for machine translation?
In total, we evaluate the conjunction {``}but{''} on 20 translation outputs, and the conjunction {``}and{''} on 10.
Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation
We reassess a recent study (Hassan et al., 2018) that claimed that machine translation (MT) has reached human parity for the translation of news from Chinese into English, using pairwise ranking and considering three variables that were not taken into account in that previous study: the language in which the source side of the test set was originally written, the translation proficiency of the evaluators, and the provision of inter-sentential context.
Using Images to Improve Machine-Translating E-Commerce Product Listings.
We evaluate our models quantitatively using BLEU and TER and find that (i) additional synthetic data has a general positive impact on text-only and multi-modal NMT models, and that (ii) using a multi-modal NMT model for re-ranking n-best lists improves TER significantly across different n-best list sizes.
Human Evaluation of Multi-modal Neural Machine Translation: A Case-Study on E-Commerce Listing Titles
Nonetheless, human evaluators ranked translations from a multi-modal NMT model as better than those of a text-only NMT over 88{\%} of the time, which suggests that images do help NMT in this use-case.
Evaluating the Impact of Light Post-Editing on Usability
This paper discusses a methodology to measure the usability of machine translated content by end users, comparing lightly post-edited content with raw output and with the usability of source language content.
PET: a Tool for Post-editing and Assessing Machine Translation
Given the significant improvements in Machine Translation (MT) quality and the increasing demand for translations, post-editing of automatic translations is becoming a popular practice in the translation industry.
