Search Results for author:
Found 6 papers, 2 papers with code
Average gradient outer product as a mechanism for deep neural collapse
In this work, we provide substantial evidence that DNC formation occurs primarily through deep feature learning with the average gradient outer product (AGOP).
Gradient descent induces alignment between weights and the empirical NTK for deep non-linear networks
Prior works have identified that the gram matrices of the weights in trained neural networks of general architectures are proportional to the average gradient outer product of the model, in a statement known as the Neural Feature Ansatz (NFA).
Mechanism of feature learning in convolutional neural networks
We then demonstrate the generality of our result by using the patch-based AGOP to enable deep feature learning in convolutional kernel machines.
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
In recent years neural networks have achieved impressive results on many technological and scientific tasks.
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
``Benign overfitting'', the ability of certain algorithms to interpolate noisy training data and yet perform well out-of-sample, has been a topic of considerable recent interest.
Learning to Hash Robustly, Guaranteed
In this paper, we design an NNS algorithm for the Hamming space that has worst-case guarantees essentially matching that of theoretical algorithms, while optimizing the hashing to the structure of the dataset (think instance-optimal algorithms) for performance on the minimum-performing query.
