TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Multimodal Machine Translation	Multi30K	VAG-NMT	BLEU (EN-DE)	31.6	# 12
Multimodal Machine Translation	Multi30K	VAG-NMT	Meteor (EN-DE)	52.2	# 11
Multimodal Machine Translation	Multi30K	VAG-NMT	Meteor (EN-FR)	70.3	# 4

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/a-visual-attention-grounding-neural-model-for/multimodal-machine-translation-on-multi30k)](https://paperswithcode.com/sota/multimodal-machine-translation-on-multi30k?p=a-visual-attention-grounding-neural-model-for)`

A Visual Attention Grounding Neural Model for Multimodal Machine Translation

EMNLP 2018 · Mingyang Zhou, Runxiang Cheng, Yong Jae Lee, Zhou Yu ·

We introduce a novel multimodal machine translation model that utilizes parallel visual and textual information. Our model jointly optimizes the learning of a shared visual-language embedding and a translator. The model leverages a visual attention grounding mechanism that links the visual semantics with the corresponding textual semantics. Our approach achieves competitive state-of-the-art results on the Multi30K and the Ambiguous COCO datasets. We also collected a new multilingual multimodal product description dataset to simulate a real-world international online shopping scenario. On this dataset, our visual attention grounding model outperforms other methods by a large margin.