Pre-trained word embeddings learned from unlabeled text have become a standard component of neural network architectures for NLP tasks. However, in most cases, the recurrent network that operates on word-level representations to produce context sensitive representations is trained on relatively little labeled data. In this paper, we demonstrate a general semi-supervised approach for adding pre- trained context embeddings from bidirectional language models to NLP systems and apply it to sequence labeling tasks.
|Task||Dataset||Model||Metric name||Metric value||Global rank||Compare|
|Named Entity Recognition||CoNLL 2003 (English)||Peters et al.||F1||91.93||# 7|