Search Results for author:
Found 17 papers, 6 papers with code
Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features
We investigate the test risk of continuous-time stochastic gradient flow dynamics in learning theory.
Gradient flow on extensive-rank positive semi-definite matrix denoising
In this work, we present a new approach to analyze the gradient flow for a positive semi-definite matrix denoising problem in an extensive-rank and high-dimensional regime.
Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structures
Even the least-squares regression has shown atypical features such as the model-wise double descent, and further works have observed triple or multiple descents.
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model
A recent line of research has highlighted that random matrix tools can be used to obtain precise analytical asymptotics of the generalization (and training) errors of the random feature model.
Statistical limits of dictionary learning: random matrix theory and the spectral replica method
We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size.
Mismatched Estimation of rank-one symmetric matrices under Gaussian noise
We consider the estimation of an n-dimensional vector s from the noisy element-wise measurements of $\mathbf{s}\mathbf{s}^T$, a generic problem that arises in statistics and machine learning.
Rank-one matrix estimation: analytic time evolution of gradient descent dynamics
Explicit formulas for the whole time evolution of the overlap between the estimator and unknown vector, as well as the cost, are rigorously derived.
Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes
The main idea is to construct a deep network which is trained from the samples of discrete stochastic differential equations underlying Kolmogorov's equation.
Information theoretic limits of learning a sparse rule
We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal.
All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation
We determine statistical and computational limits for estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix, in a sparse limit, where the underlying hidden vector (that constructs the rank-one matrix) has a number of non-zero components that scales sub-linearly with the total dimension of the vector, and the signal-to-noise ratio tends to infinity at an appropriate speed.
Bell Diagonal and Werner state generation: entanglement, non-locality, steering and discord on the IBM quantum computer
We propose the first correct special-purpose quantum circuits for preparation of Bell-diagonal states (BDS), and implement them on the IBM Quantum computer, characterizing and testing complex aspects of their quantum correlations in the full parameter space.
Quantum Physics Information Theory Information Theory
0-1 phase transitions in sparse spiked matrix estimation
We consider statistical models of estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix in the sparse limit.
Ranked #1 on
Person Re-Identification
on Market-1501
(Average-mAP metric)
Rank-one matrix estimation: analysis of algorithmic and information theoretic limits by the spatial coupling method
We characterize the detectability phase transitions in a large set of estimation problems, where we show that there exists a gap between what currently known polynomial algorithms (in particular spectral methods and approximate message-passing) can do and what is expected information theoretically.
The committee machine: Computational to statistical gaps in learning a two-layers neural network
Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks.
Entropy and mutual information in models of deep neural networks
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities.
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
Non-rigorous predictions for the optimal errors existed for special cases of GLMs, e. g. for the perceptron, in the field of statistical physics based on the so-called replica method.
Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
We also show that for a large set of parameters, an iterative algorithm called approximate message-passing is Bayes optimal.
