Search Results for author:
Found 6 papers, 1 papers with code
A new role for circuit expansion for learning in neural networks
We find that sparse expansion of the input of a student perceptron network both increases its capacity and improves the generalization performance of the network when learning a noisy rule from a teacher perceptron when these expansions are pruned after learning.
Dimensionality compression and expansion in Deep Neural Networks
Datasets such as images, text, or movies are embedded in high-dimensional spaces.
Minnorm training: an algorithm for training over-parameterized deep neural networks
To solve this constrained optimization problem, our method employs Lagrange multipliers that act as integrators of error over training and identify `support vector'-like examples.
On the Information Bottleneck Theory of Deep Learning
The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior.
An equivalence between high dimensional Bayes optimal inference and M-estimation
In this work we demonstrate, when the signal distribution and the likelihood function associated with the noise are both log-concave, that optimal MMSE performance is asymptotically achievable via another M-estimation procedure.
Statistical Mechanics of High-Dimensional Inference
Our analysis uncovers fundamental limits on the accuracy of inference in high dimensions, and reveals that widely cherished inference algorithms like maximum likelihood (ML) and maximum-a posteriori (MAP) inference cannot achieve these limits.
