Search Results for author:
Found 39 papers, 15 papers with code
BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP
Encoder-based transformer models are central to biomedical and clinical Natural Language Processing (NLP), as their bidirectional self-attention makes them well-suited for efficiently extracting structured information from unstructured text through discriminative tasks.
Exploring Scaling Laws for EHR Foundation Models
The emergence of scaling laws has profoundly shaped the development of large language models (LLMs), enabling predictable performance gains through systematic increases in model size, dataset volume, and compute.
X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains
Experiments show that X-Reasoner successfully transfers reasoning capabilities to both multimodal and out-of-domain settings, outperforming existing state-of-the-art models trained with in-domain and multimodal data across various general and medical benchmarks (Figure 1).
Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning
We introduce Med-RLVR as an initial study of RLVR in the medical domain leveraging medical multiple-choice question answering (MCQA) data as verifiable labels.
BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once
On object recognition, which aims to identify all objects in a given image along with their semantic types, we showed that BiomedParse can simultaneously segment and label all biomedical objects in an image (all at once).
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Frontier general-domain models such as GPT-4V still have significant performance gaps in multimodal biomedical applications.
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries
Summarizing clinical text is crucial in health decision-support and clinical research.
DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generation
Medical text generation aims to assist with administrative work and highlight salient information to support decision-making.
TRIALSCOPE: A Unifying Causal Framework for Scaling Real-World Evidence Generation with Biomedical Language Models
The rapid digitization of real-world data offers an unprecedented opportunity for optimizing healthcare delivery and accelerating biomedical discovery.
Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology
Clinical trial matching is a key process in health delivery and discovery.
Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events
We find that while LLMs already possess decent competency in structuring biomedical text, by distillation into a task-specific student model through self-supervised learning, substantial gains can be attained over out-of-box LLMs, with additional advantages such as cost, efficiency, and white-box model access.
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
In this paper, we propose a cost-efficient approach for training a vision-language conversational assistant that can answer open-ended research questions of biomedical images.
Self-Verification Improves Few-Shot Clinical Information Extraction
Extracting patient information from unstructured text is a critical task in health decision-support and clinical research.
Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
These findings showcase the utility of SUFO in enhancing trust and safety when using transformers in medicine, and we believe SUFO can aid practitioners in evaluating fine-tuned language models for other applications in medicine and in more critical domains.
What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarization
Summarization models often generate text that is poorly calibrated to quality metrics because they are trained to maximize the likelihood of a single reference (MLE).
Compositional Zero-Shot Domain Transfer with Text-to-Text Models
We evaluate DoT5 on the biomedical domain and the resource-lean subdomain of radiology, focusing on NLI, text summarisation and embedding learning.
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs
Therefore, training an effective generalist biomedical model requires high-quality multimodal data, such as parallel image-text pairs.
Ranked #2 on
Medical Visual Question Answering
on SLAKE-English
Continual Contrastive Finetuning Improves Low-Resource Relation Extraction
In this paper, we aim at bridging the gap and propose to pretrain and finetune the RE model using consistent objectives of contrastive learning.
Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing
We release a new dataset with locally-aligned phrase grounding annotations by radiologists to facilitate the study of complex semantic modelling in biomedical vision--language processing.
A collection of invited non-archival papers for the Conference on Health, Inference, and Learning (CHIL) 2022
A collection of invited non-archival papers for the Conference on Health, Inference, and Learning (CHIL) 2022.
Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision
Results: We conduct an extensive study on 135, 107 patients from the cancer registry of a large integrated delivery network (IDN) comprising healthcare systems in five western US states.
Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing
Overall, domainspecific vocabulary and pretraining facilitate more robust models for fine-tuning.
Knowledge-Rich Self-Supervision for Biomedical Entity Linking
Zero-shot entity linking has emerged as a promising direction for generalizing to new entities, but it still requires example gold entity mentions during training and canonical descriptions for all entities, both of which are rarely available outside of Wikipedia.
Modular Self-Supervision for Document-Level Relation Extraction
Extracting relations across large text spans has been relatively underexplored in NLP, but it is particularly important for high-value domains such as biomedicine, where obtaining high recall of the latest findings is crucial for practical applications.
Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
A prominent case in point is the explosion of the biomedical literature on COVID-19, which swelled to hundreds of thousands of papers in a matter of months.
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
In this paper, we challenge this assumption by showing that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.
Ranked #2 on
Participant Intervention Comparison Outcome Extraction
on EBM-NLP
(using extra training data)
ML4H Abstract Track 2019
A collection of the accepted abstracts for the Machine Learning for Health (ML4H) workshop at NeurIPS 2019.
Cross-Language Aphasia Detection using Optimal Transport Domain Adaptation
We learn mappings from other languages to English and detect aphasia from linguistic characteristics of speech, and show that OT domain adaptation improves aphasia detection over unilingual baselines for French (6% increased F1) and Mandarin (5% increased F1).
Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks
When training clinical prediction models from electronic health records (EHRs), a key concern should be a model's ability to sustain performance over time when deployed, even as care practices, database systems, and population demographics evolve.
MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III
Robust machine learning relies on access to data that can be used with standardized frameworks in important tasks and the ability to develop models whose performance can be reasonably reproduced.
Ranked #3 on
Length-of-Stay prediction
on MIMIC-III
Publicly Available Clinical BERT Embeddings
Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months.
Generalizability of predictive models for intensive care unit patients
A large volume of research has considered the creation of predictive models for clinical data; however, much existing literature reports results using only a single source of data.
Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation
Machine learning for healthcare often trains models on de-identified datasets with randomly-shifted calendar dates, ignoring the fact that data were generated under hospital operation practices that change over time.
Machine Learning for Health (ML4H) Workshop at NeurIPS 2018
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
Natural Language Processing for EHR-Based Computational Phenotyping
This article reviews recent advances in applying natural language processing (NLP) to Electronic Health Records (EHRs) for computational phenotyping.
A Review of Challenges and Opportunities in Machine Learning for Health
Modern electronic health records (EHRs) provide data to answer clinically meaningful questions.
Towards the Creation of a Large Corpus of Synthetically-Identified Clinical Notes
Clinical notes often describe the most important aspects of a patient's physiology and are therefore critical to medical research.
CliNER 2.0: Accessible and Accurate Clinical Concept Extraction
Clinical notes often describe important aspects of a patient's stay and are therefore critical to medical research.
