HumanMT

HumanMT is a collection of human ratings and corrections of machine translations. It consists of two parts: The first part contains five-point and pairwise sentence-level ratings, the second part contains error markings and corrections. Details are described in the following.

I. Sentence-level ratings This is a collection of five-point and pairwise ratings for 1000 German-English machine translations of TED talks (IWSLT 2014). The ratings were collected with the purpose of assessing machine translation quality rating reliability and learnability to improve a neural machine translation model with human reinforcement (see publications).

II. Error markings and corrections This is a collection of word-level error markings and post-edits/corrections for 3120 English-German machine translated sentences of 30 selected TED talks (IWSLT 2017). Each sentence received either a correction or a marking of errors from human annotators. This data was collected with the purpose of comparing annotation cost and quality, and potential for downstream machine translation improvements between annotation modes (see publications).