Several mechanisms to focus attention of a neural network on selected parts of its input or memory have been used successfully in deep learning models in recent years. Attention has improved image classification, image captioning, speech recognition, generative models, and learning algorithmic tasks, but it had probably the largest impact on neural machine translation. Recently, similar improvements have been obtained using alternative mechanisms that do not focus on a single part of a memory but operate on all of it in parallel, in a uniform way.
|Task||Dataset||Model||Metric name||Metric value||Global rank||Compare|
|Machine Translation||WMT2014 English-French||GRU+Attention||BLEU score||26.4||# 27|