Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

We introduce a model for constructing vector representations of words by composing characters using bidirectional LSTMs. Relative to traditional word representation models that have independent vectors for each word type, our model requires only a single vector per character type and a fixed set of parameters for the compositional model. Despite the compactness of this model and, more importantly, the arbitrary nature of the form-function relationship in language, our "composed" word representations yield state-of-the-art results in language modeling and part-of-speech tagging. Benefits over traditional baselines are particularly pronounced in morphologically rich languages (e.g., Turkish).

PDF Abstract EMNLP 2015 PDF EMNLP 2015 Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Part-Of-Speech Tagging Penn Treebank Char Bi-LSTM Accuracy 97.78 # 4
Part-Of-Speech Tagging Penn Treebank Bi-LSTM Accuracy 97.36 # 17

Methods


No methods listed for this paper. Add relevant methods here