Search Results for author:
Found 7 papers, 3 papers with code
Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons
This study aims to assess the performance of state-of-the-art LLMs on neurosurgery board-like questions and to evaluate their robustness to the inclusion of distractor statements.
Medical large language models are easily distracted
Large language models (LLMs) have the potential to transform medicine, but real-world clinical scenarios contain extraneous information that can hinder performance.
It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education
Using this benchmark, we evaluated three state-of-the-art LLMs (GPT-4o, GPT-3. 5, and LLama-3-70B-instruct) and found an average absolute deterioration of 39. 43% in performance on free-response questions relative to multiple-choice (p = 1. 3 * 10-5) which was greater than the human performance decline of 22. 29%.
Repurposing the scientific literature with vision-language models
Using NeuroPubs, VLMs generated publication-ready graphical abstracts (70% of 100 abstracts) and board-style questions indistinguishable from human-written ones (54% of 89, 587 questions).
MedG-KRP: Medical Graph Knowledge Representation Probing
We test GPT-4, Llama3-70b, and PalmyraMed-70b, a specialized medical model.
An Analysis of Euclidean vs. Graph-Based Framing for Bilingual Lexicon Induction from Word Embedding Spaces
Alternatively, word embeddings may be understood as nodes in a weighted graph.
LqRT: Robust Hypothesis Testing of Location Parameters using Lq-Likelihood-Ratio-Type Test in Python
To the extent that the robustness of the Wilcoxon test (minimum asymptotic relative efficiency (ARE) of the Wilcoxon test vs the t-test is 0. 864) suggests that the Wilcoxon test should be the default test of choice (rather than "use Wilcoxon if there is evidence of non-normality", the default position should be "use Wilcoxon unless there is good reason to believe the normality assumption"), the results in this article suggest that the LqRT is potentially the new default go-to test for practitioners.
Methodology
