# Combining learning rate decay and weight decay with complexity gradient descent - Part I

7 Feb 2019

The role of $L^2$ regularization, in the specific case of deep neural networks rather than more traditional machine learning models, is still not fully elucidated. We hypothesize that this complex interplay is due to the combination of overparameterization and high dimensional phenomena that take place during training and make it unamenable to standard convex optimization methods... (read more)

PDF Abstract

# Code Add Remove Mark official

No code implementations yet. Submit your code now