1 code implementation • 12 Dec 2023 • Lifeng Han, Serge Gladkoff, Gleb Erofeev, Irina Sorokina, Betty Galiano, Goran Nenadic
Furthermore, to address the language resource imbalance issue, we also carry out experiments using a transfer learning methodology based on massive multilingual pre-trained language models (MMPLMs).
no code implementations • 31 Jul 2023 • Serge Gladkoff, Gleb Erofeev, Irina Sorokina, Lifeng Han, Goran Nenadic
Translation Quality Evaluation (TQE) is an essential step of the modern translation production process.
no code implementations • 8 Mar 2023 • Serge Gladkoff, Lifeng Han, Goran Nenadic
Then, this leads to our example with two human-generated observational scores, for which, we introduce ``Student's \textit{t}-Distribution'' method and explain how to use it to measure the IRR score using only these two data points, as well as the confidence intervals (CIs) of the quality evaluation.
no code implementations • 12 Oct 2022 • Lifeng Han, Gleb Erofeev, Irina Sorokina, Serge Gladkoff, Goran Nenadic
To the best of our knowledge, this is the first work on using MMPLMs towards \textit{clinical domain transfer-learning NMT} successfully for totally unseen languages during pre-training.
no code implementations • 15 Sep 2022 • Lifeng Han, Gleb Erofeev, Irina Sorokina, Serge Gladkoff, Goran Nenadic
Pre-trained language models (PLMs) often take advantage of the monolingual and multilingual dataset that is freely available online to acquire general or mixed domain knowledge before deployment into specific tasks.
1 code implementation • LREC 2022 • Serge Gladkoff, Lifeng Han
The initial experimental work carried out on English-Russian language pair MT outputs on marketing content type of text from highly technical domain reveals that our evaluation framework is quite effective in reflecting the MT output quality regarding both overall system-level performance and segment-level transparency, and it increases the IRR for error type interpretation.
no code implementations • LREC 2022 • Serge Gladkoff, Irina Sorokina, Lifeng Han, Alexandra Alekseeva
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task.
1 code implementation • WMT (EMNLP) 2021 • Lifeng Han, Irina Sorokina, Gleb Erofeev, Serge Gladkoff
Then we present the customised hLEPOR (cushLEPOR) which uses Optuna hyper-parameter optimisation framework to fine-tune hLEPOR weighting parameters towards better agreement to pre-trained language models (using LaBSE) regarding the exact MT language pairs that cushLEPOR is deployed to.