1 code implementation • 12 Jun 2024 • Jie Ruan, Xiao Pu, Mingqi Gao, Xiaojun Wan, Yuesheng Zhu
Human evaluation is viewed as a reliable evaluation method for NLG which is expensive and time-consuming.
1 code implementation • 12 Jun 2024 • Jie Ruan, Wenqing Wang, Xiaojun Wan
Human evaluation serves as the gold standard for assessing the quality of Natural Language Generation (NLG) systems.
1 code implementation • 18 Feb 2024 • Xunjian Yin, Xu Zhang, Jie Ruan, Xiaojun Wan
In recent years, substantial advancements have been made in the development of large language models, achieving remarkable performance across diverse tasks.
no code implementations • 2 Feb 2024 • Mingqi Gao, Xinyu Hu, Jie Ruan, Xiao Pu, Xiaojun Wan
Evaluating natural language generation (NLG) is a vital but challenging problem in artificial intelligence.
no code implementations • 22 Oct 2023 • Shiang Hu, Jie Ruan, Juan Hou, Pedro Antonio Valdes-Sosa, Zhao Lv
Then, the impacts on postprocessing were quantified by the deviation caused by the IPE or EPE from the CE as to the 4 temporal statistics, the multichannel power, the cross spectra, the dispersion of source imaging, and the properties of scalp EEG network.
no code implementations • 2 May 2023 • Anya Belz, Craig Thomson, Ehud Reiter, Gavin Abercrombie, Jose M. Alonso-Moral, Mohammad Arvan, Anouck Braggaar, Mark Cieliebak, Elizabeth Clark, Kees Van Deemter, Tanvi Dinkar, Ondřej Dušek, Steffen Eger, Qixiang Fang, Mingqi Gao, Albert Gatt, Dimitra Gkatzia, Javier González-Corbelle, Dirk Hovy, Manuela Hürlimann, Takumi Ito, John D. Kelleher, Filip Klubicka, Emiel Krahmer, Huiyuan Lai, Chris van der Lee, Yiru Li, Saad Mahamood, Margot Mieskes, Emiel van Miltenburg, Pablo Mosteiro, Malvina Nissim, Natalie Parde, Ondřej Plátek, Verena Rieser, Jie Ruan, Joel Tetreault, Antonio Toral, Xiaojun Wan, Leo Wanner, Lewis Watson, Diyi Yang
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
1 code implementation • 5 Apr 2023 • Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, Xiaojun Wan
Evaluating text summarization is a challenging problem, and existing evaluation metrics are far from satisfactory.
no code implementations • 20 Nov 2022 • Jie Ruan, Yue Wu, Xiaojun Wan, Yuesheng Zhu
Sarcasm generation has been investigated in previous studies by considering it as a text-to-text generation problem, i. e., generating a sarcastic sentence for an input sentence.
no code implementations • 2 Jun 2019 • Zhun Fan, Zhaojun Wang, Wenji Li, Yutong Yuan, Yugen You, Zhi Yang, Fuzan Sun, Jie Ruan, Zhaocheng Li
In dealing with constrained multi-objective optimization problems (CMOPs), a key issue of multi-objective evolutionary algorithms (MOEAs) is to balance the convergence and diversity of working populations.
no code implementations • 16 Dec 2018 • Zhun Fan, Wenji Li, Zhaojun Wang, Yutong Yuan, Fuzan Sun, Zhi Yang, Jie Ruan, Zhaocheng Li, Erik Goodman
In the top sub-population, the search process is divided into two different stages --- push and pull stages. An adaptive DE variant with three trial vector generation strategies is employed in the proposed PPS-DE.