Search Results for author:
Found 7 papers, 0 papers with code
High-dimensional SGD aligns with emerging outlier eigenspaces
We rigorously study the joint evolution of training dynamics via stochastic gradient descent (SGD) and the spectra of empirical Hessian and gradient matrices.
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling
We prove limit theorems for the trajectories of summary statistics (i. e., finite-dimensional functions) of SGD as the dimension goes to infinity.
Online stochastic gradient descent on non-convex losses from high-dimensional inference
Here one produces an estimator of an unknown parameter from independent samples of data by iteratively optimizing a loss function.
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems.
Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges.
The landscape of the spiked tensor model
We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions $n$, and give exact formulas for the exponential growth rate.
Explorations on high dimensional landscapes
Finding minima of a real valued non-convex function over a high dimensional space is a major challenge in science.
