Search Results for author: Andy Ballard

Found 2 papers, 2 papers with code

Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping

2 code implementations5 Oct 2021 James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl-Dickstein, Samuel S. Schoenholz

Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep networks that prevent them from training fast and generalizing to unseen data, and show how these can be avoided by carefully controlling the "shape" of the network's initialization-time kernel function.

Cannot find the paper you are looking for? You can Submit a new open access paper.