Search Results for author:
Found 7 papers, 2 papers with code
How isotropic kernels perform on simple invariants
(ii) Next we consider support-vector binary classification and introduce the stripe model where the data label depends on a single coordinate $y(\underline{x}) = y(x_1)$, corresponding to parallel decision boundaries separating labels of different signs, and consider that there is no margin at these interfaces.
Disentangling feature and lazy training in deep neural networks
Two distinct limits for deep learning have been derived as the network width $h\rightarrow \infty$, depending on how the weights of the last layer scale with $h$.
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
We extract $a$ from real data by performing kernel PCA, leading to $\beta\approx0. 36$ for MNIST and $\beta\approx0. 07$ for CIFAR10, in good agreement with observations.
Scaling description of generalization with number of parameters in deep learning
At this threshold, we argue that $\|f_{N}\|$ diverges.
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved.
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
In the vicinity of this transition, properties of the curvature of the minima of the loss are critical.
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems.
