LLM-Mini-CEX: Automatic Evaluation of Large Language Model for Diagnostic Conversation

no code implementations • 15 Aug 2023 • Xiaoming Shi, Jie Xu, Jinru Ding, Jiali Pang, Sichen Liu, Shuqing Luo, Xingwei Peng, Lu Lu, Haihong Yang, Mingtao Hu, Tong Ruan, Shaoting Zhang

Despite their alluring technological potential, there is no unified and comprehensive evaluation criterion, leading to the inability to evaluate the quality and potential risks of medical LLMs, further hindering the application of LLMs in medical treatment scenarios.

Language Modelling Large Language Model +1

MedGPTEval: A Dataset and Benchmark to Evaluate Responses of Large Language Models in Medicine

no code implementations • 12 May 2023 • Jie Xu, Lu Lu, Sen yang, Bilin Liang, Xinwei Peng, Jiali Pang, Jinru Ding, Xiaoming Shi, Lingrui Yang, Huan Song, Kang Li, Xin Sun, Shaoting Zhang

The responses generated by chatbots based on LLMs are recorded for blind evaluations by five licensed medical experts.

Benchmarking