Learning Rate Schedules

# Linear Warmup With Cosine Annealing

Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule afterwards.

#### Papers

Paper Code Results Date Stars