Search Results for author:
Found 5 papers, 2 papers with code
Feature emergence via margin maximization: case studies in algebraic tasks
Understanding the internal representations learned by neural networks is a cornerstone challenge in the science of machine learning.
Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
The success of SGD in deep learning has been ascribed by prior works to the implicit bias induced by high learning rate or small batch size ("SGD noise").
Feature-Learning Networks Are Consistent Across Widths At Realistic Scales
We call this the bias of narrower width.
Using noise resilience for ranking generalization of deep neural networks
Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels.
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets
We analyse both standard weight normalization (SWN) and exponential weight normalization (EWN), and show that the gradient flow path with EWN is equivalent to gradient flow on standard networks with an adaptive learning rate.
