Learning Rate Schedules

Slanted Triangular Learning Rates

Introduced by Howard et al. in Universal Language Model Fine-tuning for Text Classification

Slanted Triangular Learning Rates (STLR) is a learning rate schedule which first linearly increases the learning rate and then linearly decays it, which can be seen in Figure to the right. It is a modification of Triangular Learning Rates, with a short increase and a long decay period.

Source: Universal Language Model Fine-tuning for Text Classification

Papers


Paper Code Results Date Stars

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories