Learning Rate Schedules

# Exponential Decay

Exponential Decay is a learning rate schedule where we decay the learning rate with more iterations using an exponential function:

$$\text{lr} = \text{lr}_{0}\exp\left(-kt\right)$$

Image Credit: Suki Lau

#### Papers

Paper Code Results Date Stars