How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

NeurIPS 2019 Devansh ArpitVictor CamposYoshua Bengio

Residual networks (ResNet) and weight normalization play an important role in various deep learning applications. However, parameter initialization strategies have not been studied previously for weight normalized networks and, in practice, initialization methods designed for un-normalized networks are used as a proxy... (read more)

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.