Search Results for author: Gerard Ben Arous

Found 7 papers, 0 papers with code

High-dimensional SGD aligns with emerging outlier eigenspaces

no code implementations4 Oct 2023 Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath

We rigorously study the joint evolution of training dynamics via stochastic gradient descent (SGD) and the spectra of empirical Hessian and gradient matrices.

High-dimensional limit theorems for SGD: Effective dynamics and critical scaling

no code implementations8 Jun 2022 Gerard Ben Arous, Reza Gheissari, Aukosh Jagannath

We prove limit theorems for the trajectories of summary statistics (i. e., finite-dimensional functions) of SGD as the dimension goes to infinity.

Vocal Bursts Intensity Prediction

Online stochastic gradient descent on non-convex losses from high-dimensional inference

no code implementations23 Mar 2020 Gerard Ben Arous, Reza Gheissari, Aukosh Jagannath

Here one produces an estimator of an unknown parameter from independent samples of data by iteratively optimizing a loss function.

General Classification Retrieval +1

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

no code implementations ICML 2018 Marco Baity-Jesi, Levent Sagun, Mario Geiger, Stefano Spigler, Gerard Ben Arous, Chiara Cammarota, Yann Lecun, Matthieu Wyart, Giulio Biroli

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems.

The landscape of the spiked tensor model

no code implementations15 Nov 2017 Gerard Ben Arous, Song Mei, Andrea Montanari, Mihai Nica

We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions $n$, and give exact formulas for the exponential growth rate.

Explorations on high dimensional landscapes

no code implementations20 Dec 2014 Levent Sagun, V. Ugur Guney, Gerard Ben Arous, Yann Lecun

Finding minima of a real valued non-convex function over a high dimensional space is a major challenge in science.

Vocal Bursts Intensity Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.