Search Results for author:
Found 37 papers, 17 papers with code
On the Origins of Linear Representations in Large Language Models
To that end, we introduce a simple latent variable model to abstract and formalize the concept dynamics of the next token prediction.
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
In this work, we relate these two approaches and study how to learn human-interpretable concepts from data.
Optimal estimation of Gaussian (poly)trees
We develop optimal algorithms for learning undirected Gaussian trees and directed Gaussian polytrees from data.
Inconsistency of cross-validation for structure learning in Gaussian graphical models
In this paper, we highlight the inherent limitations of cross-validation when employed to discern the structure of a Gaussian graphical model.
iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models
Structural causal models (SCMs) are widely used in various disciplines to represent causal relationships among variables in complex systems.
Learning Linear Causal Representations from Interventions under General Nonlinear Mixing
We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function is completely general.
Neuro-Causal Factor Analysis
Factor analysis (FA) is a statistical tool for studying how observed variables with some mutual dependences can be expressed as functions of mutually independent unobserved factors, and it is widely applied throughout the psychological, biological, and physical sciences.
Optimizing NOTEARS Objectives via Topological Swaps
In this work, we delve into the optimization challenges associated with this class of non-convex programs.
Learning Mixtures of Gaussians with Censored Data
We study the problem of learning mixtures of Gaussians with censored data.
DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization
From the optimization side, we drop the typically used augmented Lagrangian scheme and propose DAGMA ($\textit{DAGs via M-matrices for Acyclicity}$), a method that resembles the central path for barrier methods.
Identifiability of deep generative models without auxiliary information
We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders that are commonly used in practice.
A non-graphical representation of conditional independence via the neighbourhood lattice
We introduce and study the neighbourhood lattice decomposition of a distribution, which is a compact, non-graphical representation of conditional independence that is valid in the absence of a faithful graphical representation.
Tight Bounds on the Hardness of Learning Simple Nonparametric Mixtures
Combining these bounds, we conclude that the optimal sample complexity of this problem properly lies in between polynomial and exponential, which is not common in learning theory.
Optimal estimation of Gaussian DAG models
In other words, at least for Gaussian models with equal error variances, learning a directed graphical model is statistically no more difficult than learning an undirected graphical model.
Tradeoffs of Linear Mixed Models in Genome-wide Association Studies
Motivated by empirical arguments that are well-known from the genome-wide association studies (GWAS) literature, we study the statistical properties of linear mixed models (LMMs) applied to GWAS.
NOTMAD: Estimating Bayesian Networks with Sample-Specific Structures and Parameters
We encode the acyclicity constraint as a smooth regularization loss which is back-propagated to the mixing function; in this way, NOTMAD shares information between context-specific acyclic graphs, enabling the estimation of Bayesian network structures and parameters at even single-sample resolution.
Efficient Bayesian network structure learning via local Markov boundary search
Perhaps surprisingly, we show that for certain graph ensembles, a simple forward greedy search algorithm (i. e. without a backward pruning phase) suffices to learn the Markov boundary of each node.
Structure learning in polynomial time: Greedy algorithms, Bregman information, and exponential families
Greedy algorithms have long been a workhorse for learning graphical models, and more broadly for learning statistical models with sparse structure.
Uniform Consistency in Nonparametric Mixture Models
We study uniform consistency in nonparametric mixture models as well as closely related mixture of regression (also known as mixed regression) models, where the regression functions are allowed to be nonparametric and the error distributions are assumed to be convolutions of a Gaussian density.
Learning latent causal graphs via mixture oracles
We study the problem of reconstructing a causal graphical model from data in the presence of latent variables.
Fundamental Limits and Tradeoffs in Invariant Representation Learning
A wide range of machine learning applications such as privacy-preserving learning, algorithmic fairness, and domain adaptation/generalization among others, involve learning invariant representations of the data that aim to achieve two competing goals: (a) maximize information or accuracy with respect to a target response, and (b) maximize invariance or independence with respect to a set of protected features (e. g., for fairness, privacy, etc).
A polynomial-time algorithm for learning nonparametric causal graphs
We establish finite-sample guarantees for a polynomial-time algorithm for learning a nonlinear, nonparametric directed acyclic graphical (DAG) model from data.
DYNOTEARS: Structure Learning from Time-Series Data
Compared to state-of-the-art methods for learning dynamic Bayesian networks, our method is both scalable and accurate on real data.
Automated Dependence Plots
To address these drawbacks, we formalize a method for automating the selection of interesting PDPs and extend PDPs beyond showing single features to show the model response along arbitrary directions, for example in raw feature space or a latent space arising from some generative model.
Globally optimal score-based learning of directed acyclic graphs in high-dimensions
We prove that $\Omega(s\log p)$ samples suffice to learn a sparse Gaussian directed acyclic graph (DAG) from data, where $s$ is the maximum Markov blanket size.
Learning Sample-Specific Models with Low-Rank Personalized Regression
Modern applications of machine learning (ML) deal with increasingly heterogeneous datasets comprised of data collected from overlapping latent subpopulations.
Learning Sparse Nonparametric DAGs
We develop a framework for learning sparse nonparametric directed acyclic graphs (DAGs) from data.
On perfectness in Gaussian graphical models
Knowing when a graphical model is perfect to a distribution is essential in order to relate separation in the graph to conditional independence in the distribution, and this is particularly important when performing inference from data.
The Sample Complexity of Semi-Supervised Learning with Nonparametric Mixture Models
We study the sample complexity of semi-supervised learning (SSL) and introduce new assumptions based on the mismatch between a mixture model learned from unlabeled data and the true mixture model induced by the (unknown) class conditional distributions.
Fault Tolerance in Iterative-Convergent Machine Learning
In this paper, we develop a general framework to quantify the effects of calculation errors on iterative-convergent algorithms and use this framework to design new strategies for checkpoint-based fault tolerance.
Sample Complexity of Nonparametric Semi-Supervised Learning
We study the sample complexity of semi-supervised learning (SSL) and introduce new assumptions based on the mismatch between a mixture model learned from unlabeled data and the true mixture model induced by the (unknown) class conditional distributions.
DAGs with NO TEARS: Continuous Optimization for Structure Learning
This is achieved by a novel characterization of acyclicity that is not only smooth but also exact.
Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering
Motivated by problems in data clustering, we establish general conditions under which families of nonparametric mixture models are identifiable, by introducing a novel framework involving clustering overfitted \emph{parametric} (i. e. misspecified) mixture models.
The neighborhood lattice for encoding partial correlations in a Hilbert space
We study the computational complexity of computing these structures and show that under a sparsity assumption, they can be computed in polynomial time, even in the absence of the assumption of perfectness to a graph.
Learning Large-Scale Bayesian Networks with the sparsebn Package
To meet this challenge, we have developed a new R package called sparsebn for learning the structure of large, sparse graphical models with a focus on Bayesian networks.
Learning Directed Acyclic Graphs with Penalized Neighbourhood Regression
We study a family of regularized score-based estimators for learning the structure of a directed acyclic graph (DAG) for a multivariate normal distribution from high-dimensional data with $p\gg n$.
Concave Penalized Estimation of Sparse Gaussian Bayesian Networks
We develop a penalized likelihood estimation framework to estimate the structure of Gaussian Bayesian networks from observational data.
