Search Results for author:
Found 7 papers, 4 papers with code
Towards a theory of how the structure of language is acquired by deep neural networks
We conjecture that the relationship between training set size and effective range of correlations holds beyond our synthetic datasets.
How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model
The model is a classification task where each class corresponds to a group of high-level features, chosen among several equivalent groups associated with the same class.
Kernels, Data & Physics
Lecture notes from the course given by Professor Julia Kempe at the summer school "Statistical physics of Machine Learning" in Les Houches.
How deep convolutional neural networks lose spatial information with training
Here, we (i) show empirically for various architectures that stability to image diffeomorphisms is achieved by both spatial and channel pooling, (ii) introduce a model scale-detection task which reproduces our empirical observations on spatial pooling and (iii) compute analitically how the sensitivity to diffeomorphisms and noise scales with depth due to spatial pooling.
What Can Be Learnt With Wide Convolutional Neural Networks?
Interestingly, we find that, despite their hierarchical structure, the functions generated by infinitely-wide deep CNNs are too rich to be efficiently learnable in high dimension.
Learning sparse features can lead to overfitting in neural networks
It is widely believed that the success of deep networks lies in their ability to learn a meaningful representation of the features of the data.
Locality defeats the curse of dimensionality in convolutional teacher-student scenarios
Convolutional neural networks perform a local and translationally-invariant treatment of the data: quantifying which of these two aspects is central to their success remains a challenge.
