Incorporating a Local Translation Mechanism into Non-autoregressive Translation

EMNLP 2020  ·  Xiang Kong, Zhisong Zhang, Eduard Hovy ·

In this work, we introduce a novel local autoregressive translation (LAT) mechanism into non-autoregressive translation (NAT) models so as to capture local dependencies among tar-get outputs. Specifically, for each target decoding position, instead of only one token, we predict a short sequence of tokens in an autoregressive way. We further design an efficient merging algorithm to align and merge the out-put pieces into one final output sequence. We integrate LAT into the conditional masked language model (CMLM; Ghazvininejad et al.,2019) and similarly adopt iterative decoding. Empirical results on five translation tasks show that compared with CMLM, our method achieves comparable or better performance with fewer decoding iterations, bringing a 2.5xspeedup. Further analysis indicates that our method reduces repeated translations and performs better at longer sentences.

PDF Abstract EMNLP 2020 PDF EMNLP 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Machine Translation WMT2014 English-German CMLM+LAT+1 iterations BLEU score 25.20 # 57
Hardware Burden None # 1
Operations per network pass None # 1
Machine Translation WMT2014 English-German CMLM+LAT+4 iterations BLEU score 27.35 # 45
Hardware Burden None # 1
Operations per network pass None # 1
Machine Translation WMT2014 German-English CMLM+LAT+4 iterations BLEU score 32.04 # 5
Machine Translation WMT2014 German-English CMLM+LAT+1 iterations BLEU score 29.91 # 8
Machine Translation WMT2016 English-Romanian CMLM+LAT+1 iterations BLEU score 30.74 # 6
Machine Translation WMT2016 English-Romanian CMLM+LAT+4 iterations BLEU score 32.87 # 2
Machine Translation WMT2016 Romanian-English CMLM+LAT+1 iterations BLEU score 31.24 # 11
Machine Translation WMT2016 Romanian-English CMLM+LAT+4 iterations BLEU score 33.26 # 5


No methods listed for this paper. Add relevant methods here