Semi-Supervised Sequence Modeling with Cross-View Training

Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. However, the supervised models only learn from task-specific labeled data during the main training phase... (read more)

PDF Abstract EMNLP 2018 PDF EMNLP 2018 Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
CCG Supertagging CCGBank CVT + Multi-task + Large Accuracy 96.1 # 1
Named Entity Recognition CoNLL 2003 (English) CVT + Multi-Task + Large F1 92.61 # 17
Named Entity Recognition CoNLL 2003 (English) CVT + Multi-Task F1 92.61 # 17
Machine Translation IWSLT2015 English-Vietnamese CVT BLEU 29.6 # 4
Named Entity Recognition Ontonotes v5 (English) CVT + Multi-Task + Large F1 88.81 # 8
Dependency Parsing Penn Treebank CVT + Multi-Task + Large UAS 96.61 # 4
LAS 95.02 # 5
Dependency Parsing Penn Treebank CVT + Multi-Task UAS 96.44 # 5
LAS 94.83 # 6
Part-Of-Speech Tagging Penn Treebank CVT + Multi-task Accuracy 97.76 # 4

Methods used in the Paper


METHOD TYPE
Sigmoid Activation
Activation Functions
Tanh Activation
Activation Functions
LSTM
Recurrent Neural Networks
BiLSTM
Bidirectional Recurrent Neural Networks
Convolution
Convolutions
CNN BiLSTM
Bidirectional Recurrent Neural Networks
Additive Attention
Attention Mechanisms
Softmax
Output Functions
Dropout
Regularization
Cross-View Training
Word Embeddings