Search Results for author:
Found 87 papers, 45 papers with code
Vision-Language Models Do Not Understand Negation
The benchmark consists of two core tasks designed to evaluate negation understanding in diverse multimodal settings: Retrieval with Negation and Multiple Choice Questions with Negated Captions.
Generative AI in Medicine
The increased capabilities of generative AI have dramatically expanded its possible use cases in medicine.
MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations
Spurious features associated with class labels can lead image classifiers to rely on shortcuts that don't generalize well to new domains.
BendVLM: Test-Time Debiasing of Vision-Language Embeddings
Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members of various racial and gender identities.
Identifying Implicit Social Biases in Vision-Language Models
Finally, we conduct an analysis of the source of such biases, by showing that the same harmful stereotypes are also present in a large image-text dataset used to train CLIP models for examples of biases that we find.
MisinfoEval: Generative AI in the Era of "Alternative Facts"
We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users with the goal of countering misinformation by appealing to their pre-existing values.
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe
Observing that LLMs exhibit uneven confidence across the semantic representation space, we argue that examples with different confidence levels should play distinct roles during the instruction-tuning process.
FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imaging
In response, we introduce FedMedICL, a unified framework and benchmark to holistically evaluate federated medical imaging challenges, simultaneously capturing label, demographic, and temporal distribution shifts.
Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selection
Machine learning models can fail on subgroups that are underrepresented during training.
Measuring Stochastic Data Complexity with Boltzmann Influence Functions
Estimating the uncertainty of a model's prediction on a test point is a crucial part of ensuring reliability and calibration under distribution shifts.
Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation
Despite the growing popularity of methods which learn from unlabeled data, the extent to which these representations rely on spurious features for prediction is unclear.
Can AI Relate: Testing Large Language Model Response for Mental Health Support
Large language models (LLMs) are already being piloted for clinical use in hospital systems like NYU Langone, Dana-Farber and the NHS.
MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making
MDAgents achieved the best performance in seven out of ten benchmarks on tasks requiring an understanding of medical knowledge and multi-modal reasoning, showing a significant improvement of up to 4. 2% (p < 0. 05) compared to previous methods' best performances.
Application-Driven Innovation in Machine Learning
As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly important.
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
Time2Stop: Adaptive and Explainable Human-AI Loop for Smartphone Overuse Intervention
Participants preferred the adaptive interventions and rated the system highly on intervention time accuracy, effectiveness, and level of trust.
Asymmetry in Low-Rank Adapters of Foundation Models
Specifically, when updating the parameter matrices of a neural network by adding a product $BA$, we observe that the $B$ and $A$ matrices have distinct functions: $A$ extracts features from the input, while $B$ uses these features to create the desired output.
Improving Black-box Robustness with In-Context Rewriting
Machine learning models for text classification often excel on in-distribution (ID) data but struggle with unseen out-of-distribution (OOD) inputs.
Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study
Participants (N=200, healthy, female-identifying patients) were randomly assigned three clinical notes in our tool with varying levels of augmentations and answered quantitative and qualitative questions evaluating their understanding of follow-up actions.
Event-Based Contrastive Learning for Medical Time Series
In clinical practice, one often needs to identify whether a patient is at high risk of adverse outcomes after some key medical event.
The Limits of Fair Medical Imaging AI In The Wild
As artificial intelligence (AI) rapidly approaches human-level performance in medical imaging, it is crucial that it does not exacerbate or propagate healthcare disparities.
FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare
This work describes the FUTURE-AI guideline as the first international consensus framework for guiding the development and deployment of trustworthy AI tools in healthcare.
Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signals
Additionally, the supervised DML model that uses ECGs with access to 8, 172 mPCWP labels demonstrated significantly better performance on the mPCWP regression task compared to the supervised baseline.
VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception
In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment.
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
More importantly, our experiments show that instruction finetuning can significantly boost the performance of LLMs for all tasks simultaneously.
Evaluating the Impact of Social Determinants on Health Prediction in the Intensive Care Unit
Social determinants of health (SDOH) -- the conditions in which people live, grow, and age -- play a crucial role in a person's health and well-being.
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification
Data sharing is crucial for open science and reproducible research, but the legal sharing of clinical data requires the removal of protected health information from electronic health records.
AI Models Close to your Chest: Robust Federated Learning Strategies for Multi-site CT
While it is well known that population differences from genetics, sex, race, and environmental factors contribute to disease, AI studies in medicine have largely focused on locoregional patient cohorts with less diverse data sources.
Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
Since accurate estimation of MI without density information requires a sample size exponential in the true MI, we assume either a single marginal or the full joint density information is known.
Change is Hard: A Closer Look at Subpopulation Shift
Machine learning models often perform poorly on subgroups that are underrepresented in the training data.
Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learning
We find that DistDeD significantly improves over prior discovery approaches, providing indications of the risk 10 hours earlier on average as well as increasing detection by 20%.
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
We propose GRACE, a lifelong model editing method, which implements spot-fixes on streaming errors of a deployed model, ensuring minimal impact on unrelated inputs.
"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts
In this work, we introduce the problem of attributing performance differences between environments to distribution shifts in the underlying data generating mechanisms.
If Influence Functions are the Answer, Then What is the Question?
Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters.
Predicting Out-of-Domain Generalization with Neighborhood Invariance
Developing and deploying machine learning models safely depends on the ability to characterize and compare their abilities to generalize to new environments.
When Personalization Harms: Reconsidering the Use of Group Attributes in Prediction
In this work, we show models that are personalized with group attributes can reduce performance at a group level.
Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations
In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes.
The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations
Across two different blackbox model architectures and four popular explainability methods, we find that the approximation quality of explanation models, also known as the fidelity, differs significantly between subgroups.
Improving the Fairness of Chest X-ray Classifiers
We also find that methods which achieve group fairness do so by worsening performance for all groups.
Semi-Markov Offline Reinforcement Learning for Healthcare
Finally, we apply our new algorithms to a real-world offline dataset pertaining to warfarin dosing for stroke prevention and demonstrate similar results.
Learning Optimal Predictive Checklists
Our results show that our method can fit simple predictive checklists that perform well and that can easily be customized to obey a rich class of custom constraints.
Quantifying the Task-Specific Information in Text-Based Classifications
This framework allows us to compare across datasets, saying that, apart from a set of ``shortcut features'', classifying each sample in the Multi-NLI task involves around 0. 4 nats more TSI than in the Quora Question Pair.
Medical Dead-ends and Learning to Identify High-risk States and Treatments
Machine learning has successfully framed many sequential decision making problems as either supervised prediction, or optimal decision-making policy identification via reinforcement learning.
Understanding the Variance Collapse of SVGD in High Dimensions
Stein variational gradient descent (SVGD) is a deterministic inference algorithm that evolves a set of particles to fit a target distribution.
A comparison of approaches to improve worst-case predictive model performance over patient subpopulations
Predictive models for clinical outcomes that are accurate on average in a patient population may underperform drastically for some subpopulations, potentially introducing or reinforcing inequities in care access and quality.
Pulling Up by the Causal Bootstraps: Causal Data Augmentation for Pre-training Debiasing
This systematic investigation underlines the importance of accounting for the underlying data-generating mechanisms and fortifying data-preprocessing pipelines with a causal framework to develop methods robust to confounding biases.
Reading Race: AI Recognises Patient's Racial Identity In Medical Images
Methods: Using private and public datasets we evaluate: A) performance quantification of deep learning models to detect race from medical images, including the ability of these models to generalize to external environments and across multiple imaging modalities, B) assessment of possible confounding anatomic and phenotype population features, such as disease distribution and body habitus as predictors of race, and C) investigation into the underlying mechanism by which AI models can recognize race.
Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning
Finally, we propose few-shot DML as an efficient way to consistently improve generalization in response to unknown test shifts presented in ooDML.
An Empirical Framework for Domain Generalization in Clinical Settings
In this work, we benchmark the performance of eight domain generalization methods on multi-site clinical time series and medical imaging data.
An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare
Reinforcement Learning (RL) has recently been applied to sequential estimation and prediction problems identifying and developing hypothetical treatment strategies for septic patients, with a particular focus on offline learning with observational data.
Confounding Feature Acquisition for Causal Effect Estimation
Reliable treatment effect estimation from observational data depends on the availability of all confounding information.
Improving Dialogue Breakdown Detection with Semi-Supervised Learning
In ablations on DBDC4 data from 2019, our semi-supervised learning methods improve the performance of a baseline BERT model by 2\% accuracy.
Multiple Sclerosis Severity Classification From Clinical Text
Next, we present MSBC, a classifier that applies MS-BERT to generate embeddings and predict EDSS and functional subscores.
Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings
Our results highlight lesser-known limitations of methods for DP learning in health care, models that exhibit steep tradeoffs between privacy and utility, and models whose predictions are disproportionately influenced by large demographic groups in the training data.
Probabilistic Machine Learning for Healthcare
Machine learning can be used to make sense of healthcare data.
Ethical Machine Learning in Health Care
The use of machine learning (ML) in health care raises numerous ethical concerns, especially as models can amplify existing health inequities.
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness
Models that perform well on a training domain often fail to generalize to out-of-domain (OOD) examples.
S2SD: Simultaneous Similarity-based Self-Distillation for Deep Metric Learning
Deep Metric Learning (DML) provides a crucial tool for visual similarity and zero-shot applications by learning generalizing embedding spaces, although recent work in DML has shown strong performance saturation across training objectives.
Ranked #10 on
Metric Learning
on CARS196
(using extra training data)
A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
Multi-task learning (MTL) is a machine learning technique aiming to improve model performance by leveraging information across many tasks.
Uniform Priors for Data-Efficient Transfer
Deep Neural Networks have shown great promise on a variety of downstream applications; but their ability to adapt and generalize to new data and tasks remains a challenge.
CheXpert++: Approximating the CheXpert labeler for Speed,Differentiability, and Probabilistic Output
CheXpert is very useful, but is relatively computationally slow, especially when integrated with end-to-end neural pipelines, is non-differentiable so can't be used in any applications that require gradients to flow through the labeler, and does not yield probabilistic outputs, which limits our ability to improve the quality of the silver labeler through techniques such as active learning.
COVID-19 Image Data Collection: Prospective Predictions Are the Future
This dataset currently contains hundreds of frontal view X-rays and is the largest public resource for COVID-19 image and prognostic data, making it a necessary resource to develop and evaluate tools to aid in the treatment of COVID-19.
Counterfactually Guided Off-policy Transfer in Clinical Settings
Domain shift, encountered when using a trained model for a new patient population, creates significant challenges for sequential decision making in healthcare since the target domain may be both data-scarce and confounded.
Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning
In this study, we present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images.
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks.
CheXclusion: Fairness gaps in deep chest X-ray classifiers
We demonstrate that TPR disparities exist in the state-of-the-art classifiers in all datasets, for all clinical tasks, and all subgroups.
Ranked #1 on
Multi-Label Classification
on MIMIC-CXR
Cross-Language Aphasia Detection using Optimal Transport Domain Adaptation
We learn mappings from other languages to English and detect aphasia from linguistic characteristics of speech, and show that OT domain adaptation improves aphasia detection over unilingual baselines for French (6% increased F1) and Mandarin (5% increased F1).
Towards Characterizing the High-dimensional Bias of Kernel-based Particle Inference Algorithms
Particle-based inference algorithm is a promising method to efficiently generate samples for an intractable target distribution by iteratively updating a set of particles.
Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks
When training clinical prediction models from electronic health records (EHRs), a key concern should be a model's ability to sustain performance over time when deployed, even as care practices, database systems, and population demographics evolve.
MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III
Robust machine learning relies on access to data that can be used with standardized frameworks in important tasks and the ability to develop models whose performance can be reasonably reproduced.
Ranked #3 on
Length-of-Stay prediction
on MIMIC-III
Reproducibility in Machine Learning for Health
Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision.
The Cells Out of Sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifiers
Understanding if classifiers generalize to out-of-sample datasets is a central problem in machine learning.
Clinically Accurate Chest X-Ray Report Generation
The automatic generation of radiology reports given medical radiographs has significant potential to operationally and improve clinical patient care.
Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation
Machine learning for healthcare often trains models on de-identified datasets with randomly-shifted calendar dates, ignoring the fact that data were generated under hospital operation practices that change over time.
The Effect of Heterogeneous Data for Alzheimer's Disease Detection from Speech
We analyze the impact of age of the added samples and if they affect fairness in classification.
Machine Learning for Health (ML4H) Workshop at NeurIPS 2018
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
ClinicalVis: Supporting Clinical Task-Focused Design Evaluation
Making decisions about what clinical tasks to prepare for is multi-factored, and especially challenging in intensive care environments where resources must be balanced with patient needs.
Human-Computer Interaction
Racial Disparities and Mistrust in End-of-Life Care
There are established racial disparities in healthcare, including during end-of-life care, when poor communication and trust can lead to suboptimal outcomes for patients and their families.
Applications
Modeling Mistrust in End-of-Life Care
In this work, we characterize the doctor-patient relationship using a machine learning-derived trust score.
A Review of Challenges and Opportunities in Machine Learning for Health
Modern electronic health records (EHRs) provide data to answer clinically meaningful questions.
Short-term Mortality Prediction for Elderly Patients Using Medicare Claims Data
Risk prediction is central to both clinical medicine and public health.
Deep Reinforcement Learning for Sepsis Treatment
Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually.
Clinical Intervention Prediction and Understanding using Deep Networks
Real-time prediction of clinical interventions remains a challenge within intensive care units (ICUs).
Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning.
The Use of Autoencoders for Discovering Patient Phenotypes
We use autoencoders to create low-dimensional embeddings of underlying patient phenotypes that we hypothesize are a governing factor in determining how different patients will react to different interventions.
Uncovering Voice Misuse Using Symbolic Mismatch
Voice disorders affect an estimated 14 million working-aged Americans, and many more worldwide.
