Investigating the interaction between gradient-only line searches and different activation functions

Gradient-only line searches (GOLS) adaptively determine step sizes along search directions for discontinuous loss functions resulting from dynamic mini-batch sub-sampling in neural network training. Step sizes in GOLS are determined by localizing Stochastic Non-Negative Associated Gradient Projection Points (SNN-GPPs) along descent directions... (read more)

Results in Papers With Code
(↓ scroll down to see all results)