Efficient Vector Representation for Documents through Corruption

8 Jul 2017  ·  Minmin Chen ·

We present an efficient document representation learning framework, Document Vector through Corruption (Doc2VecC). Doc2VecC represents each document as a simple average of word embeddings. It ensures a representation generated as such captures the semantic meanings of the document during learning. A corruption model is included, which introduces a data-dependent regularization that favors informative or rare words while forcing the embeddings of common and non-discriminative ones to be close to zero. Doc2VecC produces significantly better word embeddings than Word2Vec. We compare Doc2VecC with several state-of-the-art document representation learning algorithms. The simple model architecture introduced by Doc2VecC matches or out-performs the state-of-the-art in generating high-quality document representations for sentiment analysis, document classification as well as semantic relatedness tasks. The simplicity of the model enables training on billions of words per hour on a single machine. At the same time, the model is very efficient in generating representations of unseen documents at test time.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Sentiment Analysis IMDb Doc2VecC Accuracy 88.3 # 37
Semantic Similarity SICK Doc2VecC MSE 0.3053 # 5
Pearson Correlation 0.8381 # 5
Spearman Correlation 0.7621 # 5


No methods listed for this paper. Add relevant methods here