Search Results for author:
Found 15 papers, 5 papers with code
How transformers learn structured data: insights from hierarchical filtering
We introduce a hierarchical filtering procedure for generative models of sequences on trees, enabling control over the range of positional correlations in the data.
Kernel Density Estimators in Large Dimensions
Our study reveals three distinct statistical regimes for the kernel-based estimate of the density $\hat \rho_h^{\mathcal {D}}(x)=\frac{1}{n h^d}\sum_{i=1}^n K\left(\frac{x-y_i}{h}\right)$, depending on the bandwidth $h$: a classical regime for large bandwidth where the Central Limit Theorem (CLT) holds, which is akin to the one found in traditional approaches.
Dynamical Regimes of Diffusion Models
Using statistical physics methods, we study generative diffusion models in the regime where the dimension of space and the number of data are large, and the score function has been trained optimally.
The Decimation Scheme for Symmetric Matrix Factorization
Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machine learning with deep networks.
Sparse Representations, Inference and Learning
In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning.
Matrix factorization with neural networks
Matrix factorization is an important mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning.
Learning curves of generic features maps for realistic datasets with a teacher-student model
While still solvable in a closed form, this generalization is able to capture the learning curves for a broad range of realistic data sets, thus redeeming the potential of the teacher-student framework.
Epidemic mitigation by statistical inference from contact tracing data
We conclude that probabilistic risk estimation is capable to enhance performance of digital contact tracing and should be considered in the currently developed mobile applications.
The Gaussian equivalence of generative models for learning with shallow neural networks
Here, we go beyond this simple paradigm by studying the performance of neural networks trained on data drawn from pre-trained generative models.
Generalisation error in learning with random features and the hidden manifold model
In particular, we show how to obtain analytically the so-called double descent behaviour for logistic regression with a peak at the interpolation threshold, we illustrate the superiority of orthogonal against random Gaussian projections in learning with random features, and discuss the role played by correlations in the data generated by the hidden manifold model.
Modelling the influence of data structure on learning in neural networks: the hidden manifold model
We demonstrate that learning of the hidden manifold model is amenable to an analytical treatment by proving a "Gaussian Equivalence Property" (GEP), and we use the GEP to show how the dynamics of two-layer neural networks trained using one-pass stochastic gradient descent is captured by a set of integro-differential equations that track the performance of the network at all times.
Multi-Layer Generalized Linear Estimation
We consider the problem of reconstructing a signal from multi-layered (possibly) non-linear measurements.
Phase transitions and sample complexity in Bayes-optimal matrix factorization
We use the tools of statistical mechanics - the cavity and replica methods - to analyze the achievability and computational tractability of the inference problems in the setting of Bayes-optimal inference, which amounts to assuming that the two matrices have random independent elements generated from some known distribution, and this information is available to the inference algorithm.
Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices
We further develop the asymptotic analysis of the corresponding phase diagrams with and without measurement noise, for different distribution of signals, and discuss the best possible reconstruction performances regardless of the algorithm.
Statistical Mechanics Information Theory Information Theory
Statistical physics-based reconstruction in compressed sensing
Compressed sensing is triggering a major evolution in signal acquisition.
Statistical Mechanics Information Theory Information Theory
