Multilingual Word Embeddings (MWEs) represent words from multiple languages
in a single distributional vector space. Unsupervised MWE (UMWE) methods
acquire multilingual embeddings without cross-lingual supervision, which is a
significant advantage over traditional supervised approaches and opens many new
possibilities for low-resource languages. Prior art for learning UMWEs,
however, merely relies on a number of independently trained Unsupervised
Bilingual Word Embeddings (UBWEs) to obtain multilingual embeddings. These
methods fail to leverage the interdependencies that exist among many languages.
To address this shortcoming, we propose a fully unsupervised framework for
learning MWEs that directly exploits the relations between all language pairs.
Our model substantially outperforms previous approaches in the experiments on
multilingual word translation and cross-lingual word similarity. In addition,
our model even beats supervised approaches trained with cross-lingual