no code implementations • 21 Feb 2024 • Zheheng Luo, Qianqian Xie, Sophia Ananiadou
Experiments on TreatFact suggest that both previous methods and LLM-based evaluators are unable to capture factual inconsistencies in clinical summaries, posing a new challenge for FC evaluation.
no code implementations • 21 Feb 2024 • Zheheng Luo, Qianqian Xie, Sophia Ananiadou
Moreover, automated methods that can effectively assess the `layness' of generated summaries are lacking.
2 code implementations • 20 Feb 2024 • Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiyuan Yao, Haohang Li, Yangyang Yu, Gang Hu, Jiajia Huang, Xiao-Yang Liu, Alejandro Lopez-Lira, Benyou Wang, Yanzhao Lai, Hao Wang, Min Peng, Sophia Ananiadou, Jimin Huang
This along with the rapid development of LLMs, highlights the urgent need for a systematic financial evaluation benchmark for LLMs.
no code implementations • 29 Sep 2023 • Tomas Goldsack, Zheheng Luo, Qianqian Xie, Carolina Scarton, Matthew Shardlow, Sophia Ananiadou, Chenghua Lin
This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm), hosted at the BioNLP Workshop at ACL 2023.
1 code implementation • 5 Jul 2023 • Zheheng Luo, Lei Liu, Qianqian Xie, Sophia Ananiadou
Based on it, we propose the graph contrastive topic model (GCTM), which conducts graph contrastive learning (GCL) using informative positive and negative samples that are generated by the graph-based sampling strategy leveraging in-depth correlation and irrelevance among documents and words.
no code implementations • 18 Apr 2023 • Qianqian Xie, Zheheng Luo, Benyou Wang, Sophia Ananiadou
In this paper, we present a systematic review of recent advancements in BTS, leveraging cutting-edge NLP techniques from PLMs to LLMs, to help understand the latest progress, challenges, and future directions.
no code implementations • 27 Mar 2023 • Zheheng Luo, Qianqian Xie, Sophia Ananiadou
In this paper, we particularly explore ChatGPT's ability to evaluate factual inconsistency under a zero-shot setting by examining it on both coarse-grained and fine-grained evaluation tasks including binary entailment inference, summary ranking, and consistency rating.
Abstractive Text Summarization Natural Language Inference +3
no code implementations • 26 Jan 2023 • Zheheng Luo, Qianqian Xie, Sophia Ananiadou
To fill that gap, we propose a novel citation-aware scientific paper summarization framework based on citation graphs, able to accurately locate and incorporate the salient contents from references, as well as capture varying relevance between source papers and their references.
no code implementations • 10 Oct 2022 • Zheheng Luo, Qianqian Xie, Sophia Ananiadou
Different from general documents, it is recognised that the ease with which people can understand a biomedical text is eminently varied, owing to the highly technical nature of biomedical documents and the variance of readers' domain knowledge.