no code implementations • 24 Dec 2024 • Yu He Ke, Liyuan Jin, Kabilan Elangovan, Bryan Wen Xi Ong, Chin Yang Oh, Jacqueline Sim, Kenny Wei-Tsen Loh, Chai Rick Soh, Jonathan Ming Hua Cheng, Aaron Kwang Yang Lee, Daniel Shu Wei Ting, Nan Liu, Hairil Rizal Abdullah
The updated PEACH demonstrated improved accuracy of 97. 9% (235/240), with a statistically significant difference from the null hypothesis of 95% accuracy (p = 0. 018, 95% CI: 0. 952-0. 991).
no code implementations • 11 Oct 2024 • Yu He Ke, Liyuan Jin, Kabilan Elangovan, Hairil Rizal Abdullah, Nan Liu, Alex Tiong Heng Sia, Chai Rick Soh, Joshua Yi Min Tung, Jasmine Chiat Ling Ong, Chang-Fu Kuo, Shao-Chun Wu, Vesela P. Kovacheva, Daniel Shu Wei Ting
The GPT4 LLM-RAG model achieved the highest accuracy (96. 4% vs. 86. 6%, p=0. 016), with no hallucinations and producing correct instructions comparable to clinicians.
no code implementations • 10 Jul 2024 • Ting Fang Tan, Kabilan Elangovan, Jasmine Ong, Nigam Shah, Joseph Sung, Tien Yin Wong, Lan Xue, Nan Liu, Haibo Wang, Chang Fu Kuo, Simon Chesterman, Zee Kin Yeong, Daniel SW Ting
A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare that expands beyond traditional accuracy and quantitative metrics needed.
no code implementations • 2 Jul 2024 • Kabilan Elangovan, Jasmine Chiat Ling Ong, Liyuan Jin, Benjamin Jun Jie Seng, Yu Heng Kwan, Lit Soo Tan, Ryan Jian Zhong, Justina Koi Li Ma, Yuhe Ke, Nan Liu, Kathleen M Giacomini, Daniel Shu Wei Ting
Mistral-7b emerged as the top performer among selected lightweight LLMs, achieving the highest median score of 14 and 71. 9% high-quality responses in accuracy and safety domains, hence chosen as the backbone LLM for Med-Pal.
no code implementations • 15 Feb 2024 • Ting Fang Tan, Kabilan Elangovan, Liyuan Jin, Yao Jie, Li Yong, Joshua Lim, Stanley Poh, Wei Yan Ng, Daniel Lim, Yuhe Ke, Nan Liu, Daniel Shu Wei Ting
200 responses to the testing dataset were generated by 5 fine-tuned LLMs for evaluation.
no code implementations • 29 Jan 2024 • Jasmine Chiat Ling Ong, Liyuan Jin, Kabilan Elangovan, Gilbert Yong San Lim, Daniel Yan Zheng Lim, Gerald Gui Ren Sng, Yuhe Ke, Joshua Yi Min Tung, Ryan Jian Zhong, Christopher Ming Yao Koh, Keane Zhi Hao Lee, Xiang Chen, Jack Kian Chng, Aung Than, Ken Junyang Goh, Daniel Shu Wei Ting
Conclusions This study established that a RAG-LLM based CDSS significantly boosts the accuracy of medication error identification when used alongside junior pharmacists (co-pilot), with notable improvements in detecting severe DRPs.
no code implementations • 29 Jan 2024 • Yuhe Ke, Liyuan Jin, Kabilan Elangovan, Hairil Rizal Abdullah, Nan Liu, Alex Tiong Heng Sia, Chai Rick Soh, Joshua Yi Min Tung, Jasmine Chiat Ling Ong, Daniel Shu Wei Ting
Compared to the human-generated instructions, which had an accuracy of 86. 3%, the performance of the GPT4. 0 RAG model demonstrated non-inferiority (p=0. 610).