Search Results for author:
Found 33 papers, 4 papers with code
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability
In particular, we study the problem of learning halfspaces under Massart noise with rate $\eta$.
Representational Power of ReLU Networks and Polynomial Kernels: Beyond Worst-Case Analysis
We give almost-tight bounds on the performance of both neural networks and low degree polynomials for this problem.
Learning Restricted Boltzmann Machines via Influence Maximization
This hardness result is based on a sharp and surprising characterization of the representational power of bounded degree RBMs: the distribution on their observed variables can simulate any bounded order MRF.
The Vertex Sample Complexity of Free Energy is Polynomial
Results in graph limit literature by Borgs, Chayes, Lov\'asz, S\'os, and Vesztergombi show that for Ising models on $n$ nodes and interactions of strength $\Theta(1/n)$, an $\epsilon$ approximation to $\log Z_n / n$ can be achieved by sampling a randomly induced model on $2^{O(1/\epsilon^2)}$ nodes.
The Mean-Field Approximation: Information Inequalities, Algorithms, and Complexity
The mean field approximation to the Ising model is a canonical variational tool that is used for analysis and inference in Ising models.
Approximating Partition Functions in Constant Time
One exception is recent results by Risteski (2016) who considered dense graphical models and showed that using variational methods, it is possible to find an $O(\epsilon n)$ additive approximation to the log partition function in time $n^{O(1/\epsilon^2)}$ even in a regime where correlation decay does not hold.
Information Theoretic Properties of Markov Random Fields, and their Algorithmic Applications
As an application, we obtain algorithms for learning Markov random fields on bounded degree graphs on $n$ nodes with $r$-order interactions in $n^r$ time and $\log n$ sample complexity.
Provable Algorithms for Inference in Topic Models
But designing provable algorithms for inference has proven to be more challenging.
Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective
More precisely, we show that the mean-field approximation is within $O((n\|J\|_{F})^{2/3})$ of the free energy, where $\|J\|_F$ denotes the Frobenius norm of the interaction matrix of the Ising model.
The Comparative Power of ReLU Networks and Polynomial Kernels in the Presence of Sparse Latent Structure
We give an almost-tight theoretical analysis of the performance of both neural networks and polynomials for this problem, as well as verify our theory with simulations.
Learning Some Popular Gaussian Graphical Models without Condition Number Bounds
While there are a variety of algorithms (e. g. Graphical Lasso, CLIME) that provably recover the graph structure with a logarithmic number of samples, they assume various conditions that require the precision matrix to be in some sense well-conditioned.
Accuracy-Memory Tradeoffs and Phase Transitions in Belief Propagation
The analysis of Belief Propagation and other algorithms for the {\em reconstruction problem} plays a key role in the analysis of community detection in inference on graphs, phylogenetic reconstruction in bioinformatics, and the cavity method in statistical physics.
Fast Convergence of Belief Propagation to Global Optima: Beyond Correlation Decay
We show that under a natural initialization, BP converges quickly to the global optimum of the Bethe free energy for Ising models on arbitrary graphs, as long as the Ising model is \emph{ferromagnetic} (i. e. neighbors prefer to be aligned).
From Boltzmann Machines to Neural Networks and Back Again
Graphical models are powerful tools for modeling high-dimensional data, but learning graphical models in the presence of latent variables is well-known to be difficult.
Representational aspects of depth and conditioning in normalizing flows
Normalizing flows are among the most popular paradigms in generative modeling, especially for images, primarily because we can efficiently evaluate the likelihood of a data point.
Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination
Our approach is based on a novel alternating minimization scheme that interleaves ordinary least-squares with a simple convex program that finds the optimal reweighting of the distribution under a spectral constraint.
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Evolvability
In this paper, we revisit the problem of distribution-independently learning halfspaces under Massart noise with rate $\eta$.
Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models
In this paper, we introduce a new algorithm that carefully combines elements of the Chow-Liu algorithm with tree metric reconstruction methods to efficiently and optimally learn tree Ising models under a prediction-centric loss.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class's Gaussian width.
On the Power of Preconditioning in Sparse Linear Regression
First, we show that the preconditioned Lasso can solve a large class of sparse linear regression problems nearly optimally: it succeeds whenever the dependency structure of the covariates, in the sense of the Markov property, has low treewidth -- even if $\Sigma$ is highly ill-conditioned.
Reconstruction on Trees and Low-Degree Polynomials
In this work, we investigate the performance of low-degree polynomials for the reconstruction problem on trees.
Multidimensional Scaling: Approximation and Complexity
In particular, the Kamada-Kawai force-directed graph drawing method is equivalent to MDS and is one of the most popular ways in practice to embed graphs into low dimensions.
Kalman Filtering with Adversarial Corruptions
In a pioneering work, Schick and Mitter gave provable guarantees when the measurement noise is a known infinitesimal perturbation of a Gaussian and raised the important question of whether one can get similar guarantees for large and unknown perturbations.
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds and Benign Overfitting
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the class’s Gaussian width.
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression
We study a localized notion of uniform convergence known as an "optimistic rate" (Panchenko 2002; Srebro et al. 2010) for linear regression with Gaussian data.
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias
Recent work by Dai and Wipf (2020) proposes a two-stage training algorithm for VAEs, based on a conjecture that in standard VAE training the generator will converge to a solution with 0 variance which is correctly supported on the ground truth manifold.
Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
We consider Ising models on the hypercube with a general interaction matrix $J$, and give a polynomial time sampling algorithm when all but $O(1)$ eigenvalues of $J$ lie in an interval of length one, a situation which occurs in many models of interest.
Distributional Hardness Against Preconditioned Lasso via Erasure-Robust Designs
Surprisingly, at the heart of our lower bound is a new positive result in compressed sensing.
Statistical Efficiency of Score Matching: The View from Isoperimetry
Roughly, we show that the score matching estimator is statistically comparable to the maximum likelihood when the distribution has a small isoperimetric constant.
A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau envelopes of the loss $\ell$.
Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initialization
There is a long history, as well as a recent explosion of interest, in statistical and generative modeling approaches based on score functions -- derivatives of the log-likelihood of a distribution.
Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps
It is well-known that the statistical performance of Lasso can suffer significantly when the covariates of interest have strong correlations.
Inferring Dynamic Networks from Marginals with Iterative Proportional Fitting
A common network inference problem, arising from real-world data constraints, is how to infer a dynamic network from its time-aggregated adjacency matrix and time-varying marginals (i. e., row and column sums).
