1 code implementation • 17 Mar 2024 • Kung Yin Hong, Lifeng Han, Riza Batista-Navarro, Goran Nenadic
We present the models we fine-tuned using the limited amount of real data and the synthetic data we generated using back-translation including OpusMT, NLLB, and mBART.
1 code implementation • 12 Dec 2023 • Lifeng Han, Serge Gladkoff, Gleb Erofeev, Irina Sorokina, Betty Galiano, Goran Nenadic
Furthermore, to address the language resource imbalance issue, we also carry out experiments using a transfer learning methodology based on massive multilingual pre-trained language models (MMPLMs).
1 code implementation • 30 Oct 2023 • Samuel Belkadi, Nicolo Micheletti, Lifeng Han, Warren Del-Pinto, Goran Nenadic
LT3 is trained on a set of around 2K lines of medication prescriptions extracted from the MIMIC-III database, allowing the model to produce valuable synthetic medication prescriptions.
no code implementations • 3 Oct 2023 • Hangyu Tu, Lifeng Han, Goran Nenadic
Furthermore, we also designed a set of post-processing roles to generate structured output on medications and the temporal relation.
1 code implementation • 22 Sep 2023 • Zihao Li, Samuel Belkadi, Nicolo Micheletti, Lifeng Han, Matthew Shardlow, Goran Nenadic
In this work, we investigate the ability of state-of-the-art large language models (LLMs) on the task of biomedical abstract simplification, using the publicly available dataset for plain language adaptation of biomedical abstracts (\textbf{PLABA}).
1 code implementation • 7 Aug 2023 • Haifa Alrdahi, Lifeng Han, Hendrik Šuvalov, Goran Nenadic
Automatic medication mining from clinical and biomedical text has become a popular topic due to its real impact on healthcare applications and the recent development of powerful language models (LMs).
no code implementations • 31 Jul 2023 • Serge Gladkoff, Gleb Erofeev, Irina Sorokina, Lifeng Han, Goran Nenadic
Translation Quality Evaluation (TQE) is an essential step of the modern translation production process.
no code implementations • 8 Mar 2023 • Serge Gladkoff, Lifeng Han, Goran Nenadic
Then, this leads to our example with two human-generated observational scores, for which, we introduce ``Student's \textit{t}-Distribution'' method and explain how to use it to measure the IRR score using only these two data points, as well as the confidence intervals (CIs) of the quality evaluation.
2 code implementations • 8 Jan 2023 • Bernadeta Griciūtė, Lifeng Han, Goran Nenadic
In this study, from the social-media and healthcare domain, we apply popular Latent Dirichlet Allocation (LDA) methods to model the topic changes in Swedish newspaper articles about Coronavirus.
no code implementations • 9 Nov 2022 • Lifeng Han
With the fast development of Machine Translation (MT) systems, especially the new boost from Neural MT (NMT) models, the MT output quality has reached a new level of accuracy.
2 code implementations • 23 Oct 2022 • Samuel Belkadi, Lifeng Han, Yuping Wu, Goran Nenadic
The experimental outcomes show that 1) CRF layers improved all language models; 2) referring to BIO-strict span level evaluation using macro-average F1 score, although the fine-tuned LLMs achieved 0. 83+ scores, the TransformerCRF model trained from scratch achieved 0. 78+, demonstrating comparable performances with much lower cost - e. g. with 39. 80\% less training parameters; 3) referring to BIO-strict span-level evaluation using weighted-average F1 score, ClinicalBERT-CRF, BERT-CRF, and TransformerCRF exhibited lower score differences, with 97. 59\%/97. 44\%/96. 84\% respectively.
no code implementations • 12 Oct 2022 • Lifeng Han, Gleb Erofeev, Irina Sorokina, Serge Gladkoff, Goran Nenadic
To the best of our knowledge, this is the first work on using MMPLMs towards \textit{clinical domain transfer-learning NMT} successfully for totally unseen languages during pre-training.
no code implementations • 15 Sep 2022 • Lifeng Han, Gleb Erofeev, Irina Sorokina, Serge Gladkoff, Goran Nenadic
Pre-trained language models (PLMs) often take advantage of the monolingual and multilingual dataset that is freely available online to acquire general or mixed domain knowledge before deployment into specific tasks.
no code implementations • 22 Feb 2022 • Lifeng Han
Manual evaluation and automatic evaluation include reference-translation based and reference-translation independent participation; automatic evaluation methods include traditional n-gram string matching, models applying syntax and semantics, and deep learning models; evaluation of evaluation methods includes estimating the credibility of human evaluations, the reliability of the automatic evaluation, the reliability of the test set, etc.
1 code implementation • LREC 2022 • Serge Gladkoff, Lifeng Han
The initial experimental work carried out on English-Russian language pair MT outputs on marketing content type of text from highly technical domain reveals that our evaluation framework is quite effective in reflecting the MT output quality regarding both overall system-level performance and segment-level transparency, and it increases the IRR for error type interpretation.
no code implementations • LREC 2022 • Serge Gladkoff, Irina Sorokina, Lifeng Han, Alexandra Alekseeva
From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task.
1 code implementation • WMT (EMNLP) 2021 • Lifeng Han, Irina Sorokina, Gleb Erofeev, Serge Gladkoff
Then we present the customised hLEPOR (cushLEPOR) which uses Optuna hyper-parameter optimisation framework to fine-tune hLEPOR weighting parameters towards better agreement to pre-trained language models (using LaBSE) regarding the exact MT language pairs that cushLEPOR is deployed to.
1 code implementation • MoTra (NoDaLiDa) 2021 • Lifeng Han, Gareth J. F. Jones, Alan F. Smeaton
To facilitate effective translation modeling and translation studies, one of the crucial questions to address is how to assess translation quality.
1 code implementation • NoDaLiDa 2021 • Lifeng Han, Gareth J. F. Jones, Alan F. Smeaton, Paolo Bolzoni
To investigate the impact of Chinese decomposition embedding in detail, i. e., radical, stroke, and intermediate levels, and how well these decompositions represent the meaning of the original character sequences, we carry out analysis with both automated and human evaluation of MT.
1 code implementation • COLING (MWE) 2020 • Lifeng Han, Gareth Jones, Alan Smeaton
To facilitate further MT research, we present a categorisation of the error types encountered by MT systems in performing MWE related translation.
1 code implementation • LREC 2020 • Lifeng Han, Gareth J. F. Jones, Alan F. Smeaton
The only bilingual MWE corpora that we are aware of is from the PARSEME (PARSing and Multi-word Expressions) EU Project.
1 code implementation • 3 May 2018 • Lifeng Han, Shaohui Kuang
We integrate the Chinese radicals into the NMT model with different settings to address the unseen words challenge in Chinese to English translation.
no code implementations • WS 2017 • Alfredo Maldonado, Lifeng Han, Erwan Moreau, Ashjan Alsulaimani, Koel Dutta Chowdhury, Carl Vogel, Qun Liu
A description of a system for identifying Verbal Multi-Word Expressions (VMWEs) in running text is presented.
1 code implementation • 26 Mar 2017 • Lifeng Han
Finally, we introduce the practical performance of our metrics in the ACL-WMT workshop shared tasks, which show that the proposed methods are robust across different languages.
no code implementations • 15 May 2016 • Lifeng Han
Subsequently, we also introduce the evaluation methods for MT evaluation including different correlation scores, and the recent quality estimation (QE) tasks for MT.