Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

Many deep learning architectures have been proposed to model the compositionality in text sequences, requiring a substantial number of parameters and expensive computations. However, there has not been a rigorous evaluation regarding the added value of sophisticated compositional functions... (read more)

PDF Abstract ACL 2018 PDF ACL 2018 Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Text Classification AG News SWEM-concat Error 7.34 # 11
Named Entity Recognition CoNLL 2000 SWEM-CRF F1 90.34 # 1
Named Entity Recognition CoNLL 2003 (English) SWEM-CRF F1 86.28 # 52
Text Classification DBpedia SWEM-concat Error 1.43 # 19
Sentiment Analysis MR SWEM-concat Accuracy 78.2 # 9
Paraphrase Identification MSRP SWEM-concat Accuracy 71.5 # 3
F1 81.3 # 3
Natural Language Inference MultiNLI SWEM-max Matched 68.2 # 29
Mismatched 67.7 # 25
Question Answering Quora Question Pairs SWEM-concat Accuracy 83.03% # 17
Natural Language Inference SNLI SWEM-max % Test Accuracy 83.8 # 78
Sentiment Analysis SST-2 Binary classification SWEM-concat Accuracy 84.3 # 57
Sentiment Analysis SST-5 Fine-grained classification SWEM-concat Accuracy 46.1 # 20
Subjectivity Analysis SUBJ SWEM-concat Accuracy 93 # 8
Text Classification TREC-6 SWEM-aver Error 7.8 # 14
Question Answering WikiQA SWEM-concat MAP 0.6788 # 13
MRR 0.6908 # 13
Text Classification Yahoo! Answers SWEM-concat Accuracy 73.53 # 8
Sentiment Analysis Yelp Binary classification SWEM-hier Error 4.19 # 15
Sentiment Analysis Yelp Fine-grained classification SWEM-hier Error 36.21 # 15

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet