Browse > Natural Language Processing > Machine Translation > Multimodal Machine Translation

# Multimodal Machine Translation Edit

7 papers with code · Natural Language Processing

Multimodal machine translation is the task of doing machine translation with multiple data sources - for example, translating "a bird is flying over water" + an image of a bird over water to German text.

No evaluation results yet. Help compare methods by submit evaluation metrics.

# On Leveraging the Visual Modality for Neural Machine Translation

7 Oct 2019

Leveraging the visual modality effectively for Neural Machine Translation (NMT) remains an open problem in computational linguistics.

# Probing Representations Learned by Multimodal Recurrent and Transformer Models

29 Aug 2019

In this paper, we present a meta-study assessing the representational quality of models where the training signal is obtained from different modalities, in particular, language modeling, image features prediction, and both textual and multimodal machine translation.

# Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation

21 Jul 2019

We present Hindi Visual Genome'', a multimodal dataset consisting of text and images suitable for English-Hindi multimodal machine translation task and multimodal research.

# Distilling Translations with Visual Awareness

Previous work on multimodal machine translation has shown that visual information is only needed in very specific cases, for example in the presence of ambiguous words where the textual context is not sufficient.

# Distilling Translations with Visual Awareness

Previous work on multimodal machine translation has shown that visual information is only needed in very specific cases, for example in the presence of ambiguous words where the textual context is not sufficient.

# Probing the Need for Visual Context in Multimodal Machine Translation

Current work on multimodal machine translation (MMT) has suggested that the visual modality is either unnecessary or only marginally beneficial.

# Multimodal Machine Translation with Embedding Prediction

Multimodal machine translation is an attractive application of neural machine translation (NMT).

# Grounded Word Sense Translation

Recent work on visually grounded language learning has focused on broader applications of grounded representations, such as visual question answering and multimodal machine translation.

# Debiasing Word Embeddings Improves Multimodal Machine Translation

24 May 2019

In this study, we examine various kinds of word embeddings and introduce two debiasing techniques for three multimodal NMT models and two language pairs -- English-German translation and English-French translation.

# Probing the Need for Visual Context in Multimodal Machine Translation

Current work on multimodal machine translation (MMT) has suggested that the visual modality is either unnecessary or only marginally beneficial.