# Mean Shift Rejection: Training Deep Neural Networks Without Minibatch Statistics or Normalization

29 Nov 2019Brendan RuffTaylor BeckJoscha Bach

Deep convolutional neural networks are known to be unstable during training at high learning rate unless normalization techniques are employed. Normalizing weights or activations allows the use of higher learning rates, resulting in faster convergence and higher test accuracy... (read more)

PDF Abstract