Search Results for author:
Found 82 papers, 35 papers with code
Last Layer Empirical Bayes
Both approaches produce predictions by computing an expectation of neural network outputs over some distribution on the corresponding weights; this distribution is given by the posterior in the case of BNNs, and by a mixture of point masses for ensembles.
Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Model
To this end, this paper proposes GrokTransfer, a simple and principled method for accelerating grokking in training neural networks, based on the key observation that data embedding plays a crucial role in determining whether generalization is delayed.
Streamlining Biomedical Research with Specialized LLMs
In this paper, we propose a novel system that integrates state-of-the-art, domain-specific large language models with advanced information retrieval techniques to deliver comprehensive and context-aware responses.
Deep Generative Models: Complexity, Dimensionality, and Approximation
Under this manifold hypothesis, it is widely believed that to approximate a distribution on a $d$-dimensional Riemannian manifold, the latent dimension needs to be at least $d$ or $d+1$.
Doubly robust identification of treatment effects from multiple environments
Practical and ethical constraints often require the use of observational data for causal inference, particularly in medicine and social sciences.
Policy Learning with a Natural Language Action Space: A Causal Approach
This paper introduces a novel causal framework for multi-stage decision-making in natural language action spaces where outcomes are only observed after a sequence of actions.
Deep Ensembles Secretly Perform Empirical Bayes
The two predominant paradigms to tackle this task are Bayesian neural networks (BNNs) and deep ensembles.
Exponential Family Attention
The self-attention mechanism is the backbone of the transformer neural network underlying most large language models.
Posterior Mean Matching: Generative Modeling through Online Bayesian Inference
We demonstrate this flexibility by developing specialized examples: a generative PMM model of real-valued data using the Normal-Normal model, a generative PMM model of count data using a Gamma-Poisson model, and a generative PMM model of discrete data using a Dirichlet-Categorical model.
Ranked #18 on
Image Generation
on CIFAR-10
Adaptive Nonparametric Perturbations of Parametric Bayesian Models
In this paper we study nonparametrically perturbed parametric (NPP) Bayesian models, in which a parametric Bayesian model is relaxed via a distortion of its likelihood.
Ordinal Preference Optimization: Aligning Human Preferences via NDCG
In this work, we propose a novel listwise approach named Ordinal Preference Optimization (OPO), which employs the Normalized Discounted Cumulative Gain (NDCG), a widely-used ranking metric, to better utilize relative proximity within ordinal multiple responses.
Brain-Cognition Fingerprinting via Graph-GCCA with Contrastive Learning
Many longitudinal neuroimaging studies aim to improve the understanding of brain aging and diseases by studying the dynamic interactions between brain function and cognition.
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
We curate an instruction-tuning dataset comprising six medical tools solving seven tasks across five modalities, enabling the agent to choose the most suitable tools for a given task.
Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisation
"Accuracy-on-the-line" is a widely observed phenomenon in machine learning, where a model's accuracy on in-distribution (ID) and out-of-distribution (OOD) data is positively correlated across different hyperparameters and data configurations.
PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry
Large language models (LLMs) have revolutionized Natural Language Processing (NLP) by minimizing the need for complex feature engineering.
Identifying Nonstationary Causal Structures with High-Order Markov Switching Models
Causal discovery in time series is a rapidly evolving field with a wide variety of applications in other areas such as climate science and neuroscience.
LLMs Are Prone to Fallacies in Causal Inference
Recent work shows that causal facts can be effectively extracted from LLMs through prompting, facilitating the creation of causal graphs for causal inference tasks.
From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When
Large language models (LLMs) like transformers demonstrate impressive in-context learning (ICL) capabilities, allowing them to make predictions for new tasks based on prompt exemplars without parameter updates.
Addressing Discretization-Induced Bias in Demographic Prediction
In particular, we show that argmax labeling, as used by a prominent commercial voter file vendor to impute race/ethnicity, results in a substantial under-count of African-American voters, e. g., by 28. 2% points in North Carolina.
PatentGPT: A Large Language Model for Intellectual Property
In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields.
Causal Inference for Human-Language Model Collaboration
A key challenge in answering this causal inference question is formulating an appropriate causal estimand: the conventional average treatment effect (ATE) estimand is inapplicable to text-based treatments due to their high dimensionality.
On the Challenges and Opportunities in Generative AI
The field of deep generative modeling has grown rapidly and consistently over the years.
Multi-Domain Causal Representation Learning via Weak Distributional Invariances
Causal representation learning has emerged as the center of action in causal machine learning research.
Imputing Brain Measurements Across Data Sets via Graph Neural Networks
An alternative is to impute the missing measurements via a deep learning approach.
The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions
This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.
Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graph
Knowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process.
Bidirectional Attention as a Mixture of Continuous Word Experts
The key observation is that fitting a single-layer single-head bidirectional attention, upon reparameterization, is equivalent to fitting a continuous bag of words (CBOW) model with mixture-of-experts (MoE) weights.
On the Identifiability of Switching Dynamical Systems
The identifiability of latent variable models has received increasing attention due to its relevance in interpretability and out-of-distribution generalisation.
Delayed and Indirect Impacts of Link Recommendations
Moreover, we show that the effects of recommendations can persist in networks, in part due to their indirect impacts on natural dynamics even after recommendations are turned off.
Clarifying Trust of Materials Property Predictions using Neural Networks with Distribution-Specific Uncertainty Quantification
It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches.
Team Resilience under Shock: An Empirical Analysis of GitHub Repositories during Early COVID-19 Pandemic
While many organizations have shifted to working remotely during the COVID-19 pandemic, how the remote workforce and the remote teams are influenced by and would respond to this and future shocks remain largely unknown.
Posterior Collapse and Latent Variable Non-identifiability
Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations.
Offline Policy Evaluation and Optimization under Confounding
Evaluating and optimizing policies in the presence of unobserved confounders is a problem of growing interest in offline reinforcement learning.
Causal Fairness Assessment of Treatment Allocation with Electronic Health Records
Healthcare continues to grapple with the persistent issue of treatment disparities, sparking concerns regarding the equitable allocation of treatments in clinical practice.
The Sample Complexity of Online Contract Design
This result shows that exponential-in-$m$ samples are sufficient and necessary to learn a near-optimal contract, resolving an open problem on the hardness of online contract design.
SAP-DETR: Bridging the Gap Between Salient Points and Queries-Based Transformer Detector for Fast Model Convergency
To bridge the gap between the reference points of salient queries and Transformer detectors, we propose SAlient Point-based DETR (SAP-DETR) by treating object detection as a transformation from salient points to instance objects.
Dynamic Survival Transformers for Causal Inference with Electronic Health Records
In medicine, researchers often seek to infer the effects of a given treatment on patients' outcomes.
Deep Learning in Single-Cell Analysis
Under each task, we describe the most recent developments in classical and deep learning methods and discuss their advantages and disadvantages.
Interventional Causal Representation Learning
Can interventional data facilitate causal representation learning?
Data-Driven Influence Functions for Optimization-Based Causal Inference
We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing, with a focus on functionals that arise in causal inference.
Anticipating Performativity by Predicting from Predictions
Predictions about people, such as their expected educational achievement or their credit risk, can be performative and shape the outcome that they aim to predict.
Valid Inference After Causal Discovery
Causal discovery and causal effect estimation are two fundamental tasks in causal inference.
Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection
Focusing on Gaussian inferential models (or variational approximating families) with diagonal plus low-rank precision matrices, we initiate a theoretical study of the trade-offs in two aspects, Bayesian posterior inference error and frequentist uncertainty quantification error.
Recommendation Systems with Distribution-Free Reliability Guarantees
Building from a pre-trained ranking model, we show how to return a set of items that is rigorously guaranteed to contain mostly good items.
Breaking Feedback Loops in Recommender Systems with Causal Inference
Our main observation is that a recommender system does not suffer from feedback loops if it reasons about causal quantities, namely the intervention distributions of recommendations on user ratings.
Partial Identification with Noisy Covariates: A Robust Optimization Approach
Directly adjusting for these imperfect measurements of the covariates can lead to biased causal estimates.
Augmenting Neural Networks with Priors on Function Values
The need for function estimation in label-limited settings is common in the natural sciences.
Spatiotemporal Clustering with Neyman-Scott Processes via Connections to Bayesian Nonparametric Mixture Models
This construction is similar to Bayesian nonparametric mixture models like the Dirichlet process mixture model (DPMM) in that the number of latent events (i. e. clusters) is a random variable, but the point process formulation makes the NSP especially well suited to modeling spatiotemporal data.
A Survey of Visual Transformers
Transformer, an attention-based encoder-decoder model, has already revolutionized the field of natural language processing (NLP).
Identifiable Deep Generative Models via Sparse Decoding
The underlying model is sparse in that each observed feature (i. e. each dimension of the data) depends on a small subset of the latent factors.
All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation
Observing this, we ask an unexplored but interesting question: can we exploit the unlabeled data via explicit real label supervision for semi-supervised training?
Optimization-based Causal Estimation from Heterogenous Environments
This paper presents a new optimization approach to causal estimation.
Desiderata for Representation Learning: A Causal Perspective
Representation learning constructs low-dimensional representations to summarize essential features of high-dimensional data.
Learning Equilibria in Matching Markets from Bandit Feedback
Large-scale, two-sided matching platforms must find market outcomes that align with user preferences while simultaneously learning these preferences from data.
ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities
Specifically, ACN adopts a novel co-training network, which enables a coupled learning process for both full modality and missing modality to supplement each other's domain and feature representations, and more importantly, to recover the `missing' information of absent modalities.
Trust It or Not: Confidence-Guided Automatic Radiology Report Generation
Experimental results have demonstrated that the proposed method for model uncertainty characterization and estimation can produce more reliable confidence scores for radiology report generation, and the modified loss function, which takes into account the uncertainties, leads to better model performance on two public radiology report datasets.
Noisy Labels are Treasure: Mean-Teacher-Assisted Confident Learning for Hepatic Vessel Segmentation
Manually segmenting the hepatic vessels from Computer Tomography (CT) is far more expertise-demanding and laborious than other structures due to the low-contrast and complex morphology of vessels, resulting in the extreme lack of high-quality labeled data.
Multi-Source Causal Inference Using Control Variates
Across simulations and two case studies with real data, we show that this control variate can significantly reduce the variance of the ATE estimate.
Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning
When a valid instrument is present, we can recover the confounded transition dynamics through observational data.
Semi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer
Accurate segmentation of cardiac structures can assist doctors to diagnose diseases, and to improve treatment planning, which is highly demanded in the clinical practice.
Modality-Pairing Learning for Brain Tumor Segmentation
In this work, we propose a novel end-to-end Modality-Pairing learning method for brain tumor segmentation.
Double-Uncertainty Weighted Method for Semi-supervised Learning
We train the teacher model using Bayesian deep learning to obtain double-uncertainty, i. e. segmentation uncertainty and feature uncertainty.
Point process models for sequence detection in high-dimensional neural spike trains
Sparse sequences of neural spikes are posited to underlie aspects of working memory, motor production, and learning.
Does Non-COVID19 Lung Lesion Help? Investigating Transferability in COVID-19 CT Image Segmentation
Coronavirus disease 2019 (COVID-19) is a highly contagious virus spreading all around the world.
Towards Data-Efficient Learning: A Benchmark for COVID-19 CT Lung and Infection Segmentation
Purpose: Accurate segmentation of lung and infection in COVID-19 CT scans plays an important role in the quantitative management of patients.
Towards Clarifying the Theory of the Deconfounder
Wang and Blei (2019) studies multiple causal inference and proposes the deconfounder algorithm.
How Distance Transform Maps Boost Segmentation CNNs: An Empirical Study
The \emph{second contribution} is that we systematically evaluated five benchmark methods on two representative public datasets.
The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge
The 2019 Kidney and Kidney Tumor Segmentation challenge (KiTS19) was a competition held in conjunction with the 2019 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) which sought to address these issues and stimulate progress on this automatic segmentation problem.
The Blessings of Multiple Causes: A Reply to Ogburn et al. (2019)
Ogburn et al. (2019, arXiv:1910. 05438) discuss "The Blessings of Multiple Causes" (Wang and Blei, 2018, arXiv:1805. 06826).
Cascaded Volumetric Convolutional Network for Kidney Tumor Segmentation from CT volumes
Automated segmentation of kidney and tumor from 3D CT scans is necessary for the diagnosis, monitoring, and treatment planning of the disease.
Multiple Causes: A Causal Graphical View
Our results expand the theory in Wang & Blei (2018), justify the deconfounder for causal graphs, and extend the settings where it can be used.
Equal Opportunity and Affirmative Action via Counterfactual Predictions
Machine learning (ML) can automate decision-making by learning to predict decisions from historical data.
Variational Bayes under Model Misspecification
As a consequence of these results, we find that the model misspecification error dominates the variational approximation error in VB posterior predictive distributions.
The Medical Deconfounder: Assessing Treatment Effects with Electronic Health Records
To adjust for unobserved confounders, we develop the medical deconfounder, a machine learning algorithm that unbiasedly estimates treatment effects from EHRs.
Using Embeddings to Correct for Unobserved Confounding in Networks
We validate the method with experiments on a semi-synthetic social network dataset.
The Deconfounded Recommender: A Causal Inference Approach to Recommendation
To this end, we develop a causal approach to recommendation, one where watching a movie is a "treatment" and a user's rating is an "outcome."
Black Box FDR
BB-FDR learns a series of black box predictive models to boost power and control the false discovery rate (FDR) at two stages of study analysis.
The Blessings of Multiple Causes
Causal inference from observational data often assumes "ignorability," that all confounders are observed.
Evaluating Bayesian Models with Posterior Dispersion Indices
We propose to evaluate a model through posterior dispersion.
Frequentist Consistency of Variational Bayes
The theorem leverages the theoretical characterizations of frequentist variational approximations to understand asymptotic properties of VB.
Minimal Dispersion Approximately Balancing Weights: Asymptotic Properties and Practical Considerations
The key observation is the connection between approximate covariate balance and shrinkage estimation of the propensity score.
Methodology Statistics Theory Applications Statistics Theory
Robust Probabilistic Modeling with Bayesian Data Reweighting
We propose a way to systematically detect and mitigate mismatch of a large class of probabilistic models.
