Search Results for author:
Found 9 papers, 2 papers with code
Learnability and Expressiveness in Self-Supervised Learning
In this work, we argue that representations induced by self-supervised learning (SSL) methods should both be expressive and learnable.
On the Regularity of Attention
Attention is a powerful component of modern neural networks across a wide variety of domains.
Implicit Regularization via Neural Feature Alignment
We approach the problem of implicit regularization in deep learning from a geometrical viewpoint.
A Mathematical Theory of Attention
Attention is a powerful component of modern neural networks across a wide variety of domains.
A Modern Take on the Bias-Variance Tradeoff in Neural Networks
The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve.
Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
On the Spectral Bias of Neural Networks
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with $100\%$ accuracy.
MINE: Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
A3T: Adversarially Augmented Adversarial Training
Recent research showed that deep neural networks are highly sensitive to so-called adversarial perturbations, which are tiny perturbations of the input data purposely designed to fool a machine learning classifier.
