Search Results for author:
Found 10 papers, 4 papers with code
Asymptotics of Learning with Deep Structured (Random) Features
For a large class of feature maps we provide a tight asymptotic characterisation of the test error associated with learning the readout layer, in the high-dimensional limit where the input dimension, hidden layer widths, and number of training samples are proportionally large.
Asymptotics of feature learning in two-layer networks after one gradient-step
To our knowledge, our results provides the first tight description of the impact of feature learning in the generalization of two-layer neural networks in the large learning rate regime $\eta=\Theta_{d}(d)$, beyond perturbative finite width corrections of the conjugate and neural tangent kernels.
A phase transition between positional and semantic learning in a solvable model of dot-product attention
We investigate how a dot-product attention layer learns a positional attention matrix (with tokens attending to each other based on their respective positions) and a semantic attention matrix (with tokens attending to each other based on their meaning).
Analysis of learning a flow-based generative model from limited sample complexity
We study the problem of training a flow-based generative model, parametrized by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture.
Deterministic equivalent and error universality of deep random features learning
Establishing this result requires proving a deterministic equivalent for traces of the deep random features sample covariance matrices which can be of independent interest.
Bayes-optimal Learning of Deep Random Networks of Extensive-width
We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
Error Scaling Laws for Kernel Classification under Source and Capacity Conditions
We find that our rates tightly describe the learning curves for this class of data sets, and are also observed on real data.
Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime
In this work, we unify and extend this line of work, providing characterization of all regimes and excess error decay rates that can be observed in terms of the interplay of noise and regularization.
Learning curves of generic features maps for realistic datasets with a teacher-student model
While still solvable in a closed form, this generalization is able to capture the learning curves for a broad range of realistic data sets, thus redeeming the potential of the teacher-student framework.
Large deviations for the perceptron model and consequences for active learning
These large deviations then provide optimal achievable performance boundaries for any active learning algorithm.
