Search Results for author:
Found 85 papers, 42 papers with code
The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers
Evaluation of large language models (LLMs) for code has primarily relied on static benchmarks, including HumanEval (Chen et al., 2021), which measure the ability of LLMs to generate complete code that passes unit tests.
Learning to Decode Collaboratively with Multiple Language Models
We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level.
Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learning
A digital twin is a virtual replica of a real-world physical phenomena that uses mathematical modeling to characterize and simulate its defining features.
A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models
In this work, we investigate the potential of large language models to generate patient summaries based on doctors' notes and study the effect of training data on the faithfulness and quality of the generated summaries.
Benchmarking Observational Studies with Experimental Data under Right-Censoring
Drawing causal inferences from observational studies (OS) requires unverifiable validity assumptions; however, one can falsify those assumptions by benchmarking the OS with experimental data from a randomized controlled trial (RCT).
Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study
Augmentations were evaluated for errors by clinicians, and we found misleading errors occur, with errors more common in real donated notes than synthetic notes, illustrating the importance of carefully written clinical notes.
Towards Verifiable Text Generation with Symbolic References
LLMs are vulnerable to hallucinations, and thus their outputs generally require laborious human verification for high-stakes applications.
Effective Human-AI Teams via Learned Natural Language Rules and Onboarding
In this work, we propose to learn rules, grounded in data regions and described in natural language, that illustrate how the human should collaborate with the AI.
Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notes
The large amount of time clinicians spend sifting through patient notes and documenting in electronic health records (EHRs) is a leading cause of clinician burnout.
Closing the Gap in High-Risk Pregnancy Care Using Machine Learning and Human-AI Collaboration
We aimed to build machine learning algorithms to identify pregnant patients and triage them by risk of complication to assist care management.
Large-Scale Study of Temporal Shift in Health Insurance Claims
Then, we construct a meta-algorithm to perform a retrospective scan for temporal shift on a large collection of tasks.
Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks
In this position paper, we argue that developing AI supports for expository writing has unique and exciting research challenges and can lead to high real-world impacts.
Conformalized Unconditional Quantile Regression
We develop a predictive inference procedure that combines conformal prediction (CP) with unconditional quantile regression (QR) -- a commonly used tool in econometrics that involves regressing the recentered influence function (RIF) of the quantile functional over input covariates.
Falsification of Internal and External Validity in Observational Studies via Conditional Moment Restrictions
Our approach is interpretable, allowing a practitioner to visualize which subgroups in the population lead to falsification of an observational study.
Who Should Predict? Exact Algorithms For Learning to Defer to Humans
We show that prior approaches can fail to find a human-AI system with low misclassification error even when there exists a linear classifier and rejector that have zero error (the realizable setting).
TabLLM: Few-shot Classification of Tabular Data with Large Language Models
We study the application of large language models to zero-shot and few-shot classification of tabular data.
Falsification before Extrapolation in Causal Effect Estimation
Under the assumption that at least one observational estimator is asymptotically normal and consistent for both the validation and extrapolated effects, we provide guarantees on the coverage probability of the intervals output by our algorithm.
Sample Efficient Learning of Predictors that Complement Humans
One of the goals of learning algorithms is to complement and reduce the burden on human decision makers.
Training Subset Selection for Weak Supervision
Subset selection applies to any label model and classifier and is very simple to plug in to existing weak supervision pipelines, requiring just a few lines of code.
Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
We give a method for proactively identifying small, plausible shifts in distribution which lead to large differences in model performance.
Large Language Models are Few-Shot Clinical Information Extractors
A long-running goal of the clinical NLP community is the extraction of important variables trapped in clinical notes.
Co-training Improves Prompt-based Learning for Large Language Models
We demonstrate that co-training (Blum & Mitchell, 1998) can improve the performance of prompt-based learning by using unlabeled data.
Teaching Humans When To Defer to a Classifier via Exemplars
For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent.
Leveraging Time Irreversibility with Order-Contrastive Pre-training
Label-scarce, high-dimensional domains such as healthcare present a challenge for modern machine learning techniques.
Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models
Our question is when using this privileged data leads to more sample-efficient learning of models that use only baseline data for predictions at test time.
Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance
Individuals often make different decisions when faced with the same context, due to personal preferences and background.
CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes
In this work, we describe our creation of a dataset of clinical action items annotated over MIMIC-III, the largest publicly available dataset of real clinical notes.
Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative
Automated decision support can accelerate tedious tasks as users can focus their attention where it is needed most.
Decision Making
Human-Computer Interaction
Regularizing towards Causal Invariance: Linear Models with Proxies
In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength.
Beyond Perturbation Stability: LP Recovery Guarantees for MAP Inference on Noisy Stable Instances
Several works have shown that perturbation stable instances of the MAP inference problem in Potts models can be solved exactly using a natural linear programming (LP) relaxation.
Neural Pharmacodynamic State Space Modeling
Modeling the time-series of high-dimensional, longitudinal data is important for predicting patient disease progression.
Clustering Interval-Censored Time-Series for Disease Phenotyping
In this work, we focus on mitigating the interference of interval censoring in the task of clustering for disease phenotyping.
Graph cuts always find a global optimum for Potts models (with a catch)
On "real-world" instances, MAP assignments of small perturbations of the problem should be very similar to the MAP assignment(s) of the original problem instance.
Trajectory Inspection: A Method for Iterative Clinician-Driven Design of Reinforcement Learning Studies
Reinforcement learning (RL) has the potential to significantly improve clinical decision making.
Robust Benchmarking for Machine Learning of Clinical Entity Extraction
We reformulate the annotation framework for clinical entity extraction to factor in these issues to allow for robust end-to-end system benchmarking.
Fast, Structured Clinical Documentation via Contextual Autocomplete
We present a system that uses a learned autocompletion mechanism to facilitate rapid creation of semi-structured clinical documentation.
PClean: Bayesian Data Cleaning at Scale with Domain-Specific Probabilistic Programming
Data cleaning is naturally framed as probabilistic inference in a generative model of ground-truth data and likely errors, but the diversity of real-world error patterns and the hardness of inference make Bayesian approaches difficult to automate.
Deep Contextual Clinical Prediction with Reverse Distillation
Healthcare providers are increasingly using machine learning to predict patient outcomes to make meaningful interventions.
Consistent Estimators for Learning to Defer to an Expert
Learning algorithms are often used in conjunction with expert decision makers in practical scenarios, however this fact is largely ignored when designing these algorithms.
Treatment Policy Learning in Multiobjective Settings with Fully Observed Outcomes
In several medical decision-making problems, such as antibiotic prescription, laboratory testing can provide precise indications for how a patient will respond to different treatment options.
Knowledge Base Completion for Constructing Problem-Oriented Medical Records
Both electronic health records and personal health records are typically organized by data type, with medical problems, medications, procedures, and laboratory results chronologically sorted in separate areas of the chart.
Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
Practitioners in diverse fields such as healthcare, economics and education are eager to apply machine learning to improve decision making.
Estimation of Bounds on Potential Outcomes For Decision Making
Estimation of individual treatment effects is commonly used as the basis for contextual decision making in fields such as healthcare, education, and economics.
Open Set Medical Diagnosis
Machine-learned diagnosis models have shown promise as medical aides but are trained under a closed-set assumption, i. e. that models will only encounter conditions on which they have been trained.
Robustly Extracting Medical Knowledge from EHRs: A Case Study of Learning a Health Knowledge Graph
Increasingly large electronic health records (EHRs) provide an opportunity to algorithmically learn medical knowledge.
Benefits of Overparameterization in Single-Layer Latent Variable Generative Models
One of the most surprising and exciting discoveries in supervising learning was the benefit of overparameterization (i. e. training a very large model) to improving the optimization landscape of a problem, with minimal effect on statistical performance (i. e. generalization).
Characterization of Overlap in Observational Studies
Overlap between treatment groups is required for non-parametric estimation of causal effects.
Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models
One of the most surprising and exciting discoveries in supervised learning was the benefit of overparameterization (i. e. training a very large model) to improving the optimization landscape of a problem, with minimal effect on statistical performance (i. e. generalization).
Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
We introduce an off-policy evaluation procedure for highlighting episodes where applying a reinforcement learned (RL) policy is likely to have produced a substantially different outcome than the observed policy.
Support and Invertibility in Domain-Invariant Representations
In this work, we give generalization bounds for unsupervised domain adaptation that hold for any representation function by acknowledging the cost of non-invertibility.
Overcomplete Independent Component Analysis via SDP
Moreover, we conjecture that the proposed program recovers a mixing component at the rate k < p^2/4 and prove that a mixing component can be recovered with high probability when k < (2 - epsilon) p log p when the original components are sampled uniformly at random on the hyper sphere.
Prototypical Clustering Networks for Dermatological Disease Diagnosis
We consider the problem of image classification for the purpose of aiding doctors in dermatological diagnosis.
Block Stability for MAP Inference
The simplest stability condition assumes that the MAP solution does not change at all when some of the pairwise potentials are (adversarially) perturbed.
Evaluating Reinforcement Learning Algorithms in Observational Health Settings
Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare.
Why Is My Classifier Discriminatory?
Recent attempts to achieve fairness in predictive models focus on the balance between fairness and accuracy.
Learning Weighted Representations for Generalization Across Designs
We pose both of these problems as prediction under a shift in design.
Semi-Amortized Variational Autoencoders
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network.
Ranked #2 on
Text Generation
on Yahoo Questions
Optimality of Approximate Inference Algorithms on Stable Instances
Approximate algorithms for structured prediction problems---such as LP relaxations and the popular alpha-expansion algorithm (Boykov et al. 2001)---typically far exceed their theoretical performance guarantees on real-world instances.
Causal Effect Inference with Deep Latent-Variable Models
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers.
Ranked #9 on
Causal Inference
on IHDP
Grounded Recurrent Neural Networks
In this work, we present the Grounded Recurrent Neural Network (GRNN), a recurrent neural network architecture for multi-label prediction which explicitly ties labels to specific dimensions of the recurrent hidden state (we call this process "grounding").
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
This work presents a novel objective function for the unsupervised training of neural network sentence encoders.
Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation
We consider multi-class classification where the predictor has a hierarchical structure that allows for a very large number of labels both at train and test time.
Structured Inference Networks for Nonlinear State Space Models
We introduce a unified algorithm to efficiently learn a broad class of linear and non-linear state space models, including variants where the emission and transition distributions are modeled by deep neural networks.
Ranked #5 on
Multivariate Time Series Forecasting
on USHCN-Daily
Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests
Disparate areas of machine learning have benefited from models that can take raw data with little preprocessing as input and learn rich representations of that raw data in order to perform well on a given prediction task.
Identifiable Phenotyping using Constrained Non-Negative Matrix Factorization
This work proposes a new algorithm for automated and simultaneous phenotyping of multiple co-occurring medical conditions, also referred as comorbidities, using clinical notes from the electronic health records (EHRs).
Clinical Tagging with Joint Probabilistic Models
We describe a method for parameter estimation in bipartite probabilistic graphical models for joint prediction of clinical conditions from the electronic medical record.
Estimating individual treatment effect: generalization bounds and algorithms
We give a novel, simple and intuitive generalization-error bound showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalization-error of that representation and the distance between the treated and control distributions induced by the representation.
Ranked #3 on
Causal Inference
on Jobs
Recurrent Neural Networks for Multivariate Time Series with Missing Values
Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values.
Ranked #4 on
Multivariate Time Series Imputation
on MuJoCo
Multivariate Time Series Forecasting
Multivariate Time Series Imputation
+3
Learning Representations for Counterfactual Inference
Observational studies are rising in importance due to the widespread accumulation of data in fields such as healthcare, education, employment and ecology.
Temporal Convolutional Neural Networks for Diagnosis from Lab Tests
Early diagnosis of treatable diseases is essential for improving healthcare, and many diseases' onsets are predictable from annual lab tests and their temporal trends.
Deep Kalman Filters
Motivated by recent variational methods for learning deep generative models, we introduce a unified algorithm to efficiently learn a broad spectrum of Kalman filters.
Anchored Discrete Factor Analysis
We present a semi-supervised learning algorithm for learning discrete factor analysis models with arbitrary structure on the latent variables.
Barrier Frank-Wolfe for Marginal Inference
We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope.
Train and Test Tightness of LP Relaxations in Structured Prediction
Structured prediction is used in areas such as computer vision and natural language processing to predict structured outputs such as segmentations or parse trees.
Character-Aware Neural Language Models
We describe a simple neural language model that relies only on character-level inputs.
Incorporating Type II Error Probabilities from Independence Tests into Score-Based Learning of Bayesian Network Structure
We give a new consistent scoring function for structure learning of Bayesian networks.
Tight Error Bounds for Structured Prediction
We show that the prospects for achieving low expected Hamming error depend on the structure of the graph $G$ in interesting ways.
Lifted Tree-Reweighted Variational Inference
We first show that for these graphical models, the tree-reweighted variational objective lends itself to a compact lifted formulation which can be solved much more efficiently than the standard TRW formulation for the ground graphical model.
Discovering Hidden Variables in Noisy-Or Networks using Quartet Tests
We show that the existence of such a quartet allows us to uniquely identify each latent variable and to learn all parameters involving that latent variable.
Unsupervised Learning of Noisy-Or Bayesian Networks
This paper considers the problem of learning the parameters in Bayesian networks of discrete variables with known structure and hidden variables.
SparsityBoost: A New Scoring Function for Learning Bayesian Network Structure
We give a new consistent scoring function for structure learning of Bayesian networks.
A Practical Algorithm for Topic Modeling with Provable Guarantees
Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora.
Complexity of Inference in Latent Dirichlet Allocation
In contrast, we show that, when a document has a large number of topics, finding the MAP assignment of topics to words in LDA is NP-hard.
More data means less inference: A pseudo-max approach to structured learning
The problem of learning to predict structured labels is of key importance in many applications.
Clusters and Coarse Partitions in LP Relaxations
We propose a new class of consistency constraints for Linear Programming (LP) relaxations for finding the most probable (MAP) configuration in graphical models.
