Universal Language Model Fine-tuning for Text Classification

ACL 2018 Jeremy HowardSebastian Ruder

Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model... (read more)

PDF Abstract

Code


Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Text Classification AG News ULMFiT Error 5.01 # 3
Text Classification DBpedia ULMFiT Error 0.80 # 4
Sentiment Analysis IMDb ULMFiT Accuracy 95.4 # 6
Text Classification TREC-6 ULMFiT Error 3.6 # 2
Sentiment Analysis Yelp Binary classification ULMFiT Error 2.16 # 3
Sentiment Analysis Yelp Fine-grained classification ULMFiT Error 29.98 # 2