Learning Rate Schedules

Slanted Triangular Learning Rates

Introduced by Howard et al. in Universal Language Model Fine-tuning for Text Classification

Slanted Triangular Learning Rates (STLR) is a learning rate schedule which first linearly increases the learning rate and then linearly decays it, which can be seen in Figure to the right. It is a modification of Triangular Learning Rates, with a short increase and a long decay period.

Source: Universal Language Model Fine-tuning for Text Classification

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Text Classification 13 11.71%
Language Modelling 13 11.71%
General Classification 13 11.71%
Language Modeling 12 10.81%
Sentiment Analysis 10 9.01%
Classification 7 6.31%
Decision Making 3 2.70%
Hate Speech Detection 3 2.70%
Management 2 1.80%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories