Search Results for author:
Found 13 papers, 4 papers with code
The unreasonable effectiveness of few-shot learning for machine translation
We demonstrate the potential of few-shot translation systems, trained with unpaired language data, for both high and low-resource language pairs.
Limitations of the NTK for Understanding Generalization in Deep Learning
The ``Neural Tangent Kernel'' (NTK) (Jacot et al 2018), and its empirical variants have been proposed as a proxy to capture certain behaviors of real neural networks.
Data Scaling Laws in NMT: The Effect of Noise and Architecture
In this work, we study the effect of varying the architecture and training data quality on the data scaling properties of Neural Machine Translation (NMT).
Distributional Generalization: Structure Beyond Test Error
Classifiers in machine learning are often reduced to single dimensional quantities, such as test error or loss.
Revisiting Model Stitching to Compare Neural Representations
We revisit and extend model stitching (Lenc & Vedaldi 2015) as a methodology to study the internal representations of neural networks.
Distributional Generalization: Characterizing Classifiers Beyond Test Error
We present a new set of empirical properties of interpolating classifiers, including neural networks, kernel machines and decision trees.
Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modelling
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the (aggregate) posterior to encourage statistical independence of the latent factors.
For self-supervised learning, Rationality implies generalization, provably
We prove a new upper bound on the generalization gap of classifiers that are obtained by first using self-supervision to learn a representation $r$ of the training data, and then fitting a simple (e. g., linear) classifier $g$ to the labels.
Distributional Generalization: A New Kind of Generalization
We introduce a new notion of generalization -- Distributional Generalization -- which roughly states that outputs of a classifier at train and test time are close *as distributions*, as opposed to close in just their average error.
CZ-GEM: A FRAMEWORK FOR DISENTANGLED REPRESENTATION LEARNING
In this work, we tackle a slightly more intricate scenario where the observations are generated from a conditional distribution of some known control variate and some latent noise variate.
Deep Double Descent: Where Bigger Models and More Data Hurt
We show that a variety of modern deep learning tasks exhibit a "double-descent" phenomenon where, as we increase model size, performance first gets worse and then gets better.
Minnorm training: an algorithm for training over-parameterized deep neural networks
To solve this constrained optimization problem, our method employs Lagrange multipliers that act as integrators of error over training and identify `support vector'-like examples.
On the Information Bottleneck Theory of Deep Learning
The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior.
