Search Results for author:
Found 20 papers, 5 papers with code
Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise
We consider a prototypical problem of Bayesian inference for a structured spiked model: a low-rank signal is corrupted by additive noise.
Fundamental limits of overparametrized shallow neural networks for supervised learning
We carry out an information-theoretical analysis of a two-layer neural network trained from input-output pairs generated by a teacher network with matching architecture, in overparametrized regimes.
Mismatched estimation of non-symmetric rank-one matrices corrupted by structured noise
We study the performance of a Bayesian statistician who estimates a rank-one signal corrupted by non-symmetric rotationally invariant noise with a generic distribution of singular values.
Bayes-optimal limits in structured PCA, and how to reach them
To answer this, we study the paradigmatic spiked matrix model of principal components analysis (PCA), where a rank-one matrix is corrupted by additive noise.
The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?
We consider the problem of estimating a rank-1 signal corrupted by structured rotationally invariant noise, and address the following question: how well do inference algorithms perform when the noise statistics is unknown and hence Gaussian noise is assumed?
Statistical limits of dictionary learning: random matrix theory and the spectral replica method
We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size.
Performance of Bayesian linear regression in a model with mismatch
Here we consider a model in which the responses are corrupted by gaussian noise and are known to be generated as linear combinations of the covariates, but the distributions of the ground-truth regression coefficients and of the noise are unknown.
High-dimensional inference: a statistical mechanics perspective
In modern signal processing and machine learning, inference is done in very high dimension: very many unknown characteristics about the system have to be deduced from a lot of high-dimensional noisy data.
Strong replica symmetry for high-dimensional disordered log-concave Gibbs measures
We consider a generic class of log-concave, possibly random, (Gibbs) measures.
Information theoretic limits of learning a sparse rule
We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal.
All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation
We determine statistical and computational limits for estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix, in a sparse limit, where the underlying hidden vector (that constructs the rank-one matrix) has a number of non-zero components that scales sub-linearly with the total dimension of the vector, and the signal-to-noise ratio tends to infinity at an appropriate speed.
Information-theoretic limits of a multiview low-rank symmetric spiked matrix model
We consider a generalization of an important class of high-dimensional inference problems, namely spiked symmetric matrix models, often used as probabilistic models for principal component analysis.
0-1 phase transitions in sparse spiked matrix estimation
We consider statistical models of estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix in the sparse limit.
Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference
We consider Bayesian inference of signals with vector-valued entries.
Overlap matrix concentration in optimal Bayesian inference
We show that, under a proper perturbation, these models are replica symmetric in the sense that the overlap matrix concentrates.
Information Theory Disordered Systems and Neural Networks Information Theory Probability
Rank-one matrix estimation: analysis of algorithmic and information theoretic limits by the spatial coupling method
We characterize the detectability phase transitions in a large set of estimation problems, where we show that there exists a gap between what currently known polynomial algorithms (in particular spectral methods and approximate message-passing) can do and what is expected information theoretically.
The committee machine: Computational to statistical gaps in learning a two-layers neural network
Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks.
Entropy and mutual information in models of deep neural networks
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities.
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
Non-rigorous predictions for the optimal errors existed for special cases of GLMs, e. g. for the perceptron, in the field of statistical physics based on the so-called replica method.
Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
We also show that for a large set of parameters, an iterative algorithm called approximate message-passing is Bayes optimal.
