Search Results for author:
Found 13 papers, 4 papers with code
From TOWER to SPIRE: Adding the Speech Modality to a Text-Only LLM
Large language models (LLMs) have shown remarkable performance and generalization capabilities across multiple languages and tasks, making them very attractive targets for multi-modality integration (e. g., images or speech).
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison
Following the remarkable success of Large Language Models (LLMs) in NLP tasks, there is increasing interest in extending their capabilities to speech -- the most common form of communication.
Findings of the IWSLT 2024 Evaluation Campaign
This paper reports on the shared tasks organized by the 21st IWSLT Conference.
Pitfalls and Outlooks in Using COMET
The COMET metric has blazed a trail in the machine translation community, given its strong correlation with human judgements of translation quality.
Compact Speech Translation Models via Discrete Speech Units Pretraining
It also avoids speech discretization in inference and is more robust to the DSU tokenization.
Prosody in Cascade and Direct Speech-to-Text Translation: a case study on Korean Wh-Phrases
While there has been a growing interest in developing direct speech translation systems to avoid propagating errors and losing non-verbal content, prior work in direct S2TT has struggled to conclusively establish the advantages of integrating the acoustic signal directly into the translation process.
Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translation
Data augmentation is a technique to generate new training data based on existing data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analyzing the Use of Influence Functions for Instance-Specific Data Filtering in Neural Machine Translation
Customer feedback can be an important signal for improving commercial machine translation systems.
Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translation
End-to-end speech translation relies on data that pair source-language speech inputs with corresponding translations into a target language.
On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR
Our method, called Aligned Data Augmentation (ADA) for ASR, replaces transcribed tokens and the speech representations in an aligned manner to generate previously unseen training pairs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Cascaded Models With Cyclic Feedback For Direct Speech Translation
Direct speech translation describes a scenario where only speech inputs and corresponding translations are available.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
We propose an interactive-predictive neural machine translation framework for easier model personalization using reinforcement and imitation learning.
A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation
We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial translations.
