no code implementations • ACL (GEM) 2021 • Lorenzo De Mattei, Huiyuan Lai, Felice Dell’Orletta, Malvina Nissim
We ask subjects whether they perceive as human-produced a bunch of texts, some of which are actually human-written, while others are automatically generated.
no code implementations • 11 Dec 2024 • Huiyuan Lai, Esther Ploeger, Rik van Noord, Antonio Toral
Neural machine translation (NMT) systems amplify lexical biases present in their training data, leading to artificially impoverished language in output translations.
no code implementations • 30 Aug 2024 • Esther Ploeger, Huiyuan Lai, Rik van Noord, Antonio Toral
Thus, rather than aiming for the rigid increase of lexical diversity, we reframe the task as recovering what is lost in the machine translation process.
no code implementations • 11 Jun 2024 • Daniela Occhipinti, Michele Marchi, Irene Mondella, Huiyuan Lai, Felice Dell'Orletta, Malvina Nissim, Marco Guerini
Results from both human and automatic evaluation show that the different quality of training data is clearly perceived and it has an impact also on the models trained on such data.
1 code implementation • 4 Jun 2024 • Huiyuan Lai, Malvina Nissim
Large language models (LLMs) with Chain-of-thought (CoT) have recently emerged as a powerful technique for eliciting reasoning to improve various downstream tasks.
1 code implementation • 1 Jun 2023 • Gosse Minnema, Huiyuan Lai, Benedetta Muscato, Malvina Nissim
Different ways of linguistically expressing the same real-world event can lead to different perceptions of what happened.
1 code implementation • 31 May 2023 • Chunliu Wang, Huiyuan Lai, Malvina Nissim, Johan Bos
Pre-trained language models (PLMs) have achieved great success in NLP and have recently been used for tasks in computational semantics.
1 code implementation • 31 May 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Figures of speech help people express abstract concepts and evoke stronger emotions than literal expressions, thereby making texts more creative and engaging.
no code implementations • 2 May 2023 • Anya Belz, Craig Thomson, Ehud Reiter, Gavin Abercrombie, Jose M. Alonso-Moral, Mohammad Arvan, Anouck Braggaar, Mark Cieliebak, Elizabeth Clark, Kees Van Deemter, Tanvi Dinkar, Ondřej Dušek, Steffen Eger, Qixiang Fang, Mingqi Gao, Albert Gatt, Dimitra Gkatzia, Javier González-Corbelle, Dirk Hovy, Manuela Hürlimann, Takumi Ito, John D. Kelleher, Filip Klubicka, Emiel Krahmer, Huiyuan Lai, Chris van der Lee, Yiru Li, Saad Mahamood, Margot Mieskes, Emiel van Miltenburg, Pablo Mosteiro, Malvina Nissim, Natalie Parde, Ondřej Plátek, Verena Rieser, Jie Ruan, Joel Tetreault, Antonio Toral, Xiaojun Wan, Leo Wanner, Lewis Watson, Diyi Yang
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
1 code implementation • 26 Apr 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We investigate the potential of ChatGPT as a multidimensional evaluator for the task of \emph{Text Style Transfer}, alongside, and in comparison to, existing automatic metrics as well as human judgements.
1 code implementation • COLING 2022 • Huiyuan Lai, Malvina Nissim
Figurative language generation is the task of reformulating a given text in the desired figure of speech while still being faithful to the original context.
1 code implementation • HumEval (ACL) 2022 • Huiyuan Lai, Jiali Mao, Antonio Toral, Malvina Nissim
Although text style transfer has witnessed rapid development in recent years, there is as yet no established standard for evaluation, which is performed using several automatic metrics, lacking the possibility of always resorting to human judgement.
1 code implementation • ACL 2022 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We exploit the pre-trained seq2seq model mBART for multilingual text style transfer.
1 code implementation • EMNLP 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Style transfer aims to rewrite a source text in a different target style while preserving its content.
1 code implementation • ACL 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content.
1 code implementation • ACL (EvalNLGEval, INLG) 2020 • Lorenzo De Mattei, Michele Cafagna, Huiyuan Lai, Felice Dell'Orletta, Malvina Nissim, Albert Gatt
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics.