Search Results for author:
Found 5 papers, 3 papers with code
Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion
In this work we explore the limiting dynamics of deep neural networks trained with stochastic gradient descent (SGD).
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion
In this work we explore the limiting dynamics of deep neural networks trained with stochastic gradient descent (SGD).
Symmetry, Conservation Laws, and Learning Dynamics in Neural Networks
Overall, by exploiting symmetry, our work demonstrates that we can analytically describe the learning dynamics of various parameter combinations at finite learning rates and batch sizes for state of the art architectures trained on any dataset.
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
Overall, by exploiting symmetry, our work demonstrates that we can analytically describe the learning dynamics of various parameter combinations at finite learning rates and batch sizes for state of the art architectures trained on any dataset.
Two Routes to Scalable Credit Assignment without Weight Symmetry
The neural plausibility of backpropagation has long been disputed, primarily for its use of non-local weight transport $-$ the biologically dubious requirement that one neuron instantaneously measure the synaptic weights of another.
