Search Results for author:
Found 9 papers, 9 papers with code
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation
Multimodal machine translation (MMT) simultaneously takes the source sentence and a relevant image as input for translation.
DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation
However, due to the presence of linguistic and acoustic diversity, the target speech follows a complex multimodal distribution, posing challenges to achieving both high-quality translations and fast decoding speeds for S2ST models.
BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models
To minimize human workload, we propose to transfer the capabilities of language generation and instruction following from English to other languages through an interactive translation task.
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation
End-to-end speech translation (ST) is the task of translating speech signals in the source language into text in the target language.
Back Translation for Speech-to-text Translation Without Transcripts
Motivated by the remarkable success of back translation in MT, we develop a back translation algorithm for ST (BT4ST) to synthesize pseudo ST data from monolingual target data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Understanding and Bridging the Modality Gap for Speech Translation
However, due to the differences between speech and text, there is always a gap between ST and MT.
Low-resource Neural Machine Translation with Cross-modal Alignment
How to achieve neural machine translation with limited parallel data?
Contrastive Learning
Low-Resource Neural Machine Translation
+2
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation
How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data?
Neural Machine Translation with Phrase-Level Universal Visual Representations
Multimodal machine translation (MMT) aims to improve neural machine translation (NMT) with additional visual information, but most existing MMT methods require paired input of source sentence and image, which makes them suffer from shortage of sentence-image pairs.
