k-decay: A New Method For Learning Rate Schedule

13 Apr 2020Tao ZhangWei Li

Recent work has shown that optimizing the learning rate (LR) schedule can be a more accurate and more efficient to train the deep neural networks. In this paper, we propose the k-decay method, in which the LR is changed by its k-th order derivative, to obtain the new LR schedule... (read more)

PDF Abstract


No code implementations yet. Submit your code now


Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.