Statistical Adaptive Stochastic Gradient Methods

We propose a statistical adaptive procedure called SALSA for automatically scheduling the learning rate (step size) in stochastic gradient methods. SALSA first uses a smoothed stochastic line-search procedure to gradually increase the learning rate, then automatically switches to a statistical method to decrease the learning rate... (read more)

Results in Papers With Code
(↓ scroll down to see all results)