Meta-Embeddings Based On Self-Attention

no code implementations • 3 Mar 2020 • Qichen Li, Yuanqing Lin, Luofeng Zhou, Jian Li

Creating meta-embeddings for better performance in language modelling has received attention lately, and methods based on concatenation or merely calculating the arithmetic mean of more than one separately trained embeddings to perform meta-embeddings have shown to be beneficial.

Language Modelling Machine Translation +3