Learning Rate Schedules
Linear Warmup
Linear Warmup is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.
Image Credit: Chengwei Zhang
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 7 | 9.21% |
| Text Generation | 5 | 6.58% |
| Language Modeling | 4 | 5.26% |
| Self-Supervised Learning | 3 | 3.95% |
| Object | 3 | 3.95% |
| Object Detection | 3 | 3.95% |
| Adversarial Robustness | 2 | 2.63% |
| Backdoor Attack | 2 | 2.63% |
| Semantic Segmentation | 2 | 2.63% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
