A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces a la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable on the fly in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the a la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.

PDF Abstract ACL 2018 PDF ACL 2018 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Sentiment Analysis CR byte mLSTM7 Accuracy 90.6 # 4
Text Classification IMDb byte mLSTM7 Accuracy (2 classes) 92.2 # 10
Accuracy (10 classes) - # 3
Sentiment Analysis MPQA byte mLSTM7 Accuracy 88.8 # 3
Sentiment Analysis MR byte mLSTM7 Accuracy 86.8 # 3
Sentiment Analysis SST-2 Binary classification byte mLSTM7 Accuracy 91.7 # 37
Sentiment Analysis SST-5 Fine-grained classification byte mLSTM7 Accuracy 54.6 # 5
Subjectivity Analysis SUBJ byte mLSTM7 Accuracy 94.7 # 8
Text Classification TREC-6 byte mLSTM7 Error 9.6 # 17

Methods


No methods listed for this paper. Add relevant methods here