Unsupervised Statistical Machine Translation

EMNLP 2018  ·  Mikel Artetxe, Gorka Labaka, Eneko Agirre ·

While modern machine translation has relied on large parallel corpora, a recent line of work has managed to train Neural Machine Translation (NMT) systems from monolingual corpora only (Artetxe et al., 2018c; Lample et al., 2018). Despite the potential of this approach for low-resource settings, existing systems are far behind their supervised counterparts, limiting their practical interest. In this paper, we propose an alternative approach based on phrase-based Statistical Machine Translation (SMT) that significantly closes the gap with supervised systems. Our method profits from the modular architecture of SMT: we first induce a phrase table from monolingual corpora through cross-lingual embedding mappings, combine it with an n-gram language model, and fine-tune hyperparameters through an unsupervised MERT variant. In addition, iterative backtranslation improves results further, yielding, for instance, 14.08 and 26.22 BLEU points in WMT 2014 English-German and English-French, respectively, an improvement of more than 7-10 BLEU points over previous unsupervised systems, and closing the gap with supervised SMT (Moses trained on Europarl) down to 2-5 BLEU points. Our implementation is available at https://github.com/artetxem/monoses

PDF Abstract EMNLP 2018 PDF EMNLP 2018 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Machine Translation WMT2014 English-French SMT + iterative backtranslation (unsupervised) BLEU score 26.22 # 50
Machine Translation WMT2014 English-German SMT + iterative backtranslation (unsupervised) BLEU score 14.08 # 69
Hardware Burden None # 1
Operations per network pass None # 1
Unsupervised Machine Translation WMT2014 French-English SMT BLEU 25.9 # 7
Machine Translation WMT2014 French-English SMT + iterative backtranslation (unsupervised) BLEU score 25.87 # 2
Machine Translation WMT2014 German-English SMT + iterative backtranslation (unsupervised) BLEU score 17.43 # 11
Machine Translation WMT2016 English-German SMT + iterative backtranslation (unsupervised) BLEU score 18.23 # 8
Machine Translation WMT2016 German-English SMT + iterative backtranslation (unsupervised) BLEU score 23.05 # 4

Methods


No methods listed for this paper. Add relevant methods here