Search Results for author:
Found 43 papers, 27 papers with code
On the Dynamics of Gender Learning in Speech Translation
In this work, we contribute to such a line of inquiry by exploring the emergence of gender bias in Speech Translation (ST).
Is “moby dick” a Whale or a Bird? Named Entities and Terminology in Speech Translation
Automatic translation systems are known to struggle with rare words.
Extending the MuST-C Corpus for a Comparative Evaluation of Speech Translation Technology
This project aimed at extending the test sets of the MuST-C speech translation (ST) corpus with new reference translations.
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison
Following the remarkable success of Large Language Models (LLMs) in NLP tasks, there is increasing interest in extending their capabilities to speech -- the most common form in communication.
Speech Foundation Models and Crowdsourcing for Efficient, High-Quality Data Collection
While crowdsourcing is an established solution for facilitating and scaling the collection of speech data, the involvement of non-experts necessitates protocols to ensure final data quality.
SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation
Spurred by the demand for interpretable models, research on eXplainable AI for language technologies has experienced significant growth, with feature attribution methods emerging as a cornerstone of this progress.
MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages
The rise of foundation models (FMs), coupled with regulatory efforts addressing their risks and impacts, has sparked significant interest in open-source models.
How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not
The remarkable performance achieved by Large Language Models (LLM) has driven research efforts to leverage them for a wide range of tasks and input modalities.
Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond
We present Speech-MASSIVE, a multilingual Spoken Language Understanding (SLU) dataset comprising the speech counterpart for a portion of the MASSIVE textual corpus.
SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation
This paper describes the FBK's participation in the Simultaneous Translation Evaluation Campaign at IWSLT 2024.
StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection
To fill this gap, we introduce StreamAtt, the first StreamST policy, and propose StreamLAAL, the first StreamST latency metric designed to be comparable with existing metrics for SimulST.
SBAAM! Eliminating Transcript Dependency in Automatic Subtitling
Subtitling plays a crucial role in enhancing the accessibility of audiovisual content and encompasses three primary subtasks: translating spoken dialogue, segmenting translations into concise textual units, and estimating timestamps that govern their on-screen duration.
How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena
The attention mechanism, a cornerstone of state-of-the-art neural models, faces computational hurdles in processing long sequences due to its quadratic complexity.
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?
The field of natural language processing (NLP) has recently witnessed a transformative shift with the emergence of foundation models, particularly Large Language Models (LLMs) that have revolutionized text-based NLP.
Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES
As part of the WMT-2023 "Test suites" shared task, in this paper we summarize the results of two test suites evaluations: MuST-SHE-WMT23 and INES.
Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection
When translating words referring to the speaker, speech translation (ST) systems should not resort to default masculine generics nor rely on potentially misleading vocal traits.
How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation
When translating from notional gender languages (e. g., English) into grammatical gender languages (e. g., Italian), the generated translation requires explicit gender assignments for various words, including those referring to the speaker.
No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulation
Automatic speech recognition (ASR) systems are known to be sensitive to the sociolinguistic variability of speech data, in which gender plays a crucial role.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023
This paper describes the FBK's participation in the Simultaneous Translation and Automatic Subtitling tracks of the IWSLT 2023 Evaluation Campaign.
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP
Despite its crucial role in research experiments, code correctness is often presumed only on the basis of the perceived quality of results.
Distributed Silhouette Algorithm: Evaluating Clustering on Big Data
To fill this gap, in this paper we introduce the first algorithm that computes the Silhouette metric with linear complexity and can easily execute in parallel in a distributed environment.
Joint Speech Translation and Named Entity Recognition
Modern automatic translation systems aim at place the human at the center by providing contextual support and knowledge.
Named Entity Detection and Injection for Direct Speech Translation
In a sentence, certain words are critical for its semantic.
Direct Speech Translation for Automatic Subtitling
Automatic subtitling is the task of automatically translating the speech of audiovisual content into short pieces of timed text, i. e. subtitles and their corresponding timestamps.
Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
Simultaneous speech translation (SimulST) systems aim at generating their output with the lowest possible latency, which is normally computed in terms of Average Lagging (AL).
Who Are We Talking About? Handling Person Names in Speech Translation
Recent work has shown that systems for speech translation (ST) -- similarly to automatic speech recognition (ASR) -- poorly handle person names.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Efficient yet Competitive Speech Translation: FBK@IWSLT2022
The primary goal of this FBK's systems submission to the IWSLT 2022 offline and simultaneous speech translation tasks is to reduce model training costs without sacrificing translation quality.
Does Simultaneous Speech Translation need Simultaneous Models?
In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task.
Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation
Gender bias is largely recognized as a problematic phenomenon affecting language technologies, with recent studies underscoring that it might surface differently across languages.
Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translation
Automatic translation systems are known to struggle with rare words.
Speechformer: Reducing Information Loss in Direct Speech Translation
Transformer-based models have gained increasing popularity achieving state-of-the-art performance in many research fields including speech translation.
Ranked #1 on
Speech-to-Text Translation
on MuST-C EN->NL
Between Flexibility and Consistency: Joint Generation of Captions and Subtitles
Speech translation (ST) has lately received growing interest for the generation of subtitles without the need for an intermediate source language transcription and timing (i. e. captions).
Dealing with training and test segmentation mismatch: FBK@IWSLT2021
Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora.
Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?
Five years after the first published proofs of concept, direct approaches to speech translation (ST) are now competing with traditional cascade solutions.
How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation
In light of this finding, we propose a combined approach that preserves BPE overall translation quality, while leveraging the higher ability of character-based segmentation to properly translate gender.
Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation
The audio segmentation mismatch between training data and those seen at run-time is a major problem in direct speech translation.
Gender Bias in Machine Translation
Machine translation (MT) technology has facilitated our daily tasks by providing accessible shortcuts for gathering, elaborating and communicating information.
CTC-based Compression for Direct Speech Translation
Previous studies demonstrated that a dynamic phone-informed compression of the input audio is beneficial for speech translation (ST).
Breeding Gender-aware Direct Speech Translation Systems
In particular, by translating speech audio data without intermediate transcription, direct ST models are able to leverage and preserve essential information present in the input (e. g. speaker's vocal characteristics) that is otherwise lost in the cascade framework.
On Knowledge Distillation for Direct Speech Translation
Direct speech translation (ST) has shown to be a complex task requiring knowledge transfer from its sub-tasks: automatic speech recognition (ASR) and machine translation (MT).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
On Target Segmentation for Direct Speech Translation
Then, subword-level segmentation became the state of the art in neural machine translation as it produces shorter sequences that reduce the training time, while being superior to word-level models.
Contextualized Translation of Automatically Segmented Speech
We show that our context-aware solution is more robust to VAD-segmented input, outperforming a strong base model and the fine-tuning on different VAD segmentations of an English-German test set by up to 4. 25 BLEU points.
End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020
The test talks are provided in two versions: one contains the data already segmented with automatic tools and the other is the raw data without any segmentation.
