Rethinking Normalization and Elimination Singularity in Neural Networks

21 Nov 2019Siyuan QiaoHuiyu WangChenxi LiuWei ShenAlan Yuille

In this paper, we study normalization methods for neural networks from the perspective of elimination singularity. Elimination singularities correspond to the points on the training trajectory where neurons become consistently deactivated... (read more)

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.