Search Results for author:
Found 7 papers, 6 papers with code
Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations
Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data.
How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model
The model is a classification task where each class corresponds to a group of high-level features, chosen among several equivalent groups associated with the same class.
How deep convolutional neural networks lose spatial information with training
Here, we (i) show empirically for various architectures that stability to image diffeomorphisms is achieved by both spatial and channel pooling, (ii) introduce a model scale-detection task which reproduces our empirical observations on spatial pooling and (iii) compute analitically how the sensitivity to diffeomorphisms and noise scales with depth due to spatial pooling.
Learning sparse features can lead to overfitting in neural networks
It is widely believed that the success of deep networks lies in their ability to learn a meaningful representation of the features of the data.
Relative stability toward diffeomorphisms indicates performance in deep nets
Understanding why deep nets can classify data in large dimensions remains a challenge.
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
In this manuscript, we review recent results elucidating (i, ii) and the perspective they offer on the (still unexplained) curse of dimensionality paradox.
Geometric compression of invariant manifolds in neural nets
We confirm these predictions both for a one-hidden layer FC network trained on the stripe model and for a 16-layers CNN trained on MNIST, for which we also find $\beta_\text{Feature}>\beta_\text{Lazy}$.
