Learning Rate Schedules

# Linear Warmup With Linear Decay

Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

#### Papers

Paper Code Results Date Stars