Semi-Supervised Sequence Modeling with Cross-View Training

EMNLP 2018 Kevin Clark • Minh-Thang Luong • Christopher D. Manning • Quoc V. Le

We therefore propose Cross-View Training (CVT), a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. On labeled examples, standard supervised learning is used. On unlabeled examples, CVT teaches auxiliary prediction modules that see restricted views of the input (e.g., only part of a sentence) to match the predictions of the full model seeing the whole input.

Full paper

Evaluation


Task Dataset Model Metric name Metric value Global rank Compare
CCG Supertagging CCGBank Clark et al. Accuracy 96.1 # 1
Named Entity Recognition CoNLL 2003 (English) CVT + Multi-Task F1 92.61 # 3
Named Entity Recognition Ontonotes v5 (English) CVT + Multi-Task F1 88.81 # 2
Dependency Parsing Penn Treebank CVT + Multi-Task POS --- # 9
Dependency Parsing Penn Treebank CVT + Multi-Task UAS 96.61 # 9
Dependency Parsing Penn Treebank CVT + Multi-Task LAS 95.02 # 9