# Luck Matters: Understanding Training Dynamics of Deep ReLU Networks

31 May 2019Yuandong TianTina JiangQucheng GongAri Morcos

We analyze the dynamics of training deep ReLU networks and their implications on generalization capability. Using a teacher-student setting, we discovered a novel relationship between the gradient received by hidden student nodes and the activations of teacher nodes for deep ReLU networks... (read more)

PDF Abstract