1 code implementation • ACL (EvalNLGEval, INLG) 2020 • Lorenzo De Mattei, Michele Cafagna, Huiyuan Lai, Felice Dell'Orletta, Malvina Nissim, Albert Gatt
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics.
1 code implementation • ACL 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content.
1 code implementation • EMNLP 2021 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Style transfer aims to rewrite a source text in a different target style while preserving its content.
1 code implementation • ACL 2022 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We exploit the pre-trained seq2seq model mBART for multilingual text style transfer.
1 code implementation • HumEval (ACL) 2022 • Huiyuan Lai, Jiali Mao, Antonio Toral, Malvina Nissim
Although text style transfer has witnessed rapid development in recent years, there is as yet no established standard for evaluation, which is performed using several automatic metrics, lacking the possibility of always resorting to human judgement.
1 code implementation • COLING 2022 • Huiyuan Lai, Malvina Nissim
Figurative language generation is the task of reformulating a given text in the desired figure of speech while still being faithful to the original context.
1 code implementation • 26 Apr 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
We investigate the potential of ChatGPT as a multidimensional evaluator for the task of \emph{Text Style Transfer}, alongside, and in comparison to, existing automatic metrics as well as human judgements.
no code implementations • 2 May 2023 • Anya Belz, Craig Thomson, Ehud Reiter, Gavin Abercrombie, Jose M. Alonso-Moral, Mohammad Arvan, Anouck Braggaar, Mark Cieliebak, Elizabeth Clark, Kees Van Deemter, Tanvi Dinkar, Ondřej Dušek, Steffen Eger, Qixiang Fang, Mingqi Gao, Albert Gatt, Dimitra Gkatzia, Javier González-Corbelle, Dirk Hovy, Manuela Hürlimann, Takumi Ito, John D. Kelleher, Filip Klubicka, Emiel Krahmer, Huiyuan Lai, Chris van der Lee, Yiru Li, Saad Mahamood, Margot Mieskes, Emiel van Miltenburg, Pablo Mosteiro, Malvina Nissim, Natalie Parde, Ondřej Plátek, Verena Rieser, Jie Ruan, Joel Tetreault, Antonio Toral, Xiaojun Wan, Leo Wanner, Lewis Watson, Diyi Yang
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
1 code implementation • 31 May 2023 • Chunliu Wang, Huiyuan Lai, Malvina Nissim, Johan Bos
Pre-trained language models (PLMs) have achieved great success in NLP and have recently been used for tasks in computational semantics.
1 code implementation • 31 May 2023 • Huiyuan Lai, Antonio Toral, Malvina Nissim
Figures of speech help people express abstract concepts and evoke stronger emotions than literal expressions, thereby making texts more creative and engaging.
1 code implementation • 1 Jun 2023 • Gosse Minnema, Huiyuan Lai, Benedetta Muscato, Malvina Nissim
Different ways of linguistically expressing the same real-world event can lead to different perceptions of what happened.
no code implementations • ACL (GEM) 2021 • Lorenzo De Mattei, Huiyuan Lai, Felice Dell’Orletta, Malvina Nissim
We ask subjects whether they perceive as human-produced a bunch of texts, some of which are actually human-written, while others are automatically generated.