Search Results for author: Qingkai Fang

Found 13 papers, 12 papers with code

CTC-based Non-autoregressive Textless Speech-to-Speech Translation

1 code implementation11 Jun 2024 Qingkai Fang, Zhengrui Ma, Yan Zhou, Min Zhang, Yang Feng

Direct speech-to-speech translation (S2ST) has achieved impressive translation quality, but it often faces the challenge of slow decoding due to the considerable length of speech sequences.

Knowledge Distillation Machine Translation +2

Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data?

no code implementations11 Jun 2024 Qingkai Fang, Shaolei Zhang, Zhengrui Ma, Min Zhang, Yang Feng

Recently proposed two-pass direct speech-to-speech translation (S2ST) models decompose the task into speech-to-text translation (S2TT) and text-to-speech (TTS) within an end-to-end model, yielding promising results.

Contrastive Learning Speech Synthesis +4

StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning

1 code implementation5 Jun 2024 Shaolei Zhang, Qingkai Fang, Shoutao Guo, Zhengrui Ma, Min Zhang, Yang Feng

Simultaneous speech-to-speech translation (Simul-S2ST, a. k. a streaming speech translation) outputs target speech while receiving streaming speech inputs, which is critical for real-time communication.

 Ranked #1 on de-en on CVSS

Automatic Speech Recognition (ASR) de-en +11

Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation

1 code implementation20 Oct 2023 Wenyu Guo, Qingkai Fang, Dong Yu, Yang Feng

Multimodal machine translation (MMT) simultaneously takes the source sentence and a relevant image as input for translation.

Decoder Multimodal Machine Translation +3

DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation

1 code implementation NeurIPS 2023 Qingkai Fang, Yan Zhou, Yang Feng

However, due to the presence of linguistic and acoustic diversity, the target speech follows a complex multimodal distribution, posing challenges to achieving both high-quality translations and fast decoding speeds for S2ST models.

Decoder fr-en +3

BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models

1 code implementation19 Jun 2023 Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, Yang Feng

To minimize human workload, we propose to transfer the capabilities of language generation and instruction following from English to other languages through an interactive translation task.

Instruction Following Text Generation +1

CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation

2 code implementations24 May 2023 Yan Zhou, Qingkai Fang, Yang Feng

End-to-end speech translation (ST) is the task of translating speech signals in the source language into text in the target language.

Machine Translation Translation

Back Translation for Speech-to-text Translation Without Transcripts

1 code implementation15 May 2023 Qingkai Fang, Yang Feng

Motivated by the remarkable success of back translation in MT, we develop a back translation algorithm for ST (BT4ST) to synthesize pseudo ST data from monolingual target data.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +5

Understanding and Bridging the Modality Gap for Speech Translation

1 code implementation15 May 2023 Qingkai Fang, Yang Feng

However, due to the differences between speech and text, there is always a gap between ST and MT.

Machine Translation Multi-Task Learning +1

Neural Machine Translation with Phrase-Level Universal Visual Representations

1 code implementation ACL 2022 Qingkai Fang, Yang Feng

Multimodal machine translation (MMT) aims to improve neural machine translation (NMT) with additional visual information, but most existing MMT methods require paired input of source sentence and image, which makes them suffer from shortage of sentence-image pairs.

Multimodal Machine Translation NMT +2

Cannot find the paper you are looking for? You can Submit a new open access paper.