Search Results for author:
Found 130 papers, 35 papers with code
Findings of the IWSLT 2022 Evaluation Campaign
The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation.
Is “moby dick” a Whale or a Bird? Named Entities and Terminology in Speech Translation
Automatic translation systems are known to struggle with rare words.
FBK’s Multilingual Neural Machine Translation System for IWSLT 2017
Particularly, we focus on the four zero-shot directions and show how a multilingual model trained with small data can provide reasonable results.
Findings of the WMT 2020 Shared Task on Automatic Post-Editing
Due to i) the different source/domain of data compared to the past (Wikipedia vs Information Technology), ii) the different quality of the initial translations to be corrected and iii) the introduction of a new language pair (English-Chinese), this year’s results are not directly comparable with last year’s round.
Zero-Shot Neural Machine Translation with Self-Learning Cycle
Neural Machine Translation (NMT) approaches employing monolingual data are showing steady improvements in resource-rich conditions.
Instance Selection for Online Automatic Post-Editing in a multi-domain scenario
In recent years, several end-to-end online translation systems have been proposed to successfully incorporate human post-editing feedback in the translation workflow.
The IWSLT 2019 Evaluation Campaign
The IWSLT 2019 evaluation campaign featured three tasks: speech translation of (i) TED talks and (ii) How2 instructional videos from English into German and Portuguese, and (iii) text translation of TED talks from English into Czech.
Findings of the 2021 Conference on Machine Translation (WMT21)
This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.
Machine-oriented NMT Adaptation for Zero-shot NLP tasks: Comparing the Usefulness of Close and Distant Languages
In this work, we tackle the problem in a multilingual setting where a single NMT model translates from multiple languages for downstream automatic processing in the target language.
FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN
The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2021) featured this year four shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Multilingual speech translation, (iv) Low-resource speech translation.
Automatic Translation for Multiple NLP tasks: a Multi-task Approach to Machine-oriented NMT Adaptation
We address this problem by proposing a multi-task approach to machine-oriented NMT adaptation, which is capable to serve multiple downstream tasks with a single system.
On the Dynamics of Gender Learning in Speech Translation
In this work, we contribute to such a line of inquiry by exploring the emergence of gender bias in Speech Translation (ST).
Extending the MuST-C Corpus for a Comparative Evaluation of Speech Translation Technology
This project aimed at extending the test sets of the MuST-C speech translation (ST) corpus with new reference translations.
Towards a methodology for evaluating automatic subtitling
In response to the increasing interest towards automatic subtitling, this EAMT-funded project aimed at collecting subtitle post-editing data in a real use case scenario where professional subtitlers edit automatically generated subtitles.
Post-editing in Automatic Subtitling: A Subtitlers’ perspective
Subtitling tools are recently being adapted for post-editing by providing automatically generated subtitles, and featuring not only machine translation, but also automatic segmentation and synchronisation.
FBK’s Neural Machine Translation Systems for IWSLT 2016
They leverage linguistic information such as lemmas and part-of-speech tags of the source words in the form of additional factors along with the words.
Data Augmentation for End-to-End Speech Translation: FBK@IWSLT ‘19
On the training side, we focused on data augmentation techniques recently proposed for ST and automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena
The attention mechanism, a cornerstone of state-of-the-art neural models, faces computational hurdles in processing long sequences due to its quadratic complexity.
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?
The field of natural language processing (NLP) has recently witnessed a transformative shift with the emergence of foundation models, particularly Large Language Models (LLMs) that have revolutionized text-based NLP.
A Prompt Response to the Demand for Automatic Gender-Neutral Translation
Gender-neutral translation (GNT) that avoids biased and undue binary assumptions is a pivotal challenge for the creation of more inclusive translation technologies.
Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES
As part of the WMT-2023 "Test suites" shared task, in this paper we summarize the results of two test suites evaluations: MuST-SHE-WMT23 and INES.
Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection
When translating words referring to the speaker, speech translation (ST) systems should not resort to default masculine generics nor rely on potentially misleading vocal traits.
How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation
When translating from notional gender languages (e. g., English) into grammatical gender languages (e. g., Italian), the generated translation requires explicit gender assignments for various words, including those referring to the speaker.
No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulation
Automatic speech recognition (ASR) systems are known to be sensitive to the sociolinguistic variability of speech data, in which gender plays a crucial role.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus
Gender inequality is embedded in our communication practices and perpetuated in translation technologies.
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023
This paper describes the FBK's participation in the Simultaneous Translation and Automatic Subtitling tracks of the IWSLT 2023 Evaluation Campaign.
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation
Attention is the core mechanism of today's most used architectures for natural language processing and has been analyzed from many perspectives, including its effectiveness for machine translation-related tasks.
Storage and Learning phase transitions in the Random-Features Hopfield Model
The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities.
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP
Despite its crucial role in research experiments, code correctness is often presumed only on the basis of the perceived quality of results.
Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges
Gender inclusivity in language technologies has become a prominent research topic.
Attention as a Guide for Simultaneous Speech Translation
The study of the attention mechanism has sparked interest in many fields, such as language modeling and machine translation.
Joint Speech Translation and Named Entity Recognition
Modern automatic translation systems aim at place the human at the center by providing contextual support and knowledge.
Multi-mode fiber reservoir computing overcomes shallow neural networks classifiers
In the field of disordered photonics, a common objective is to characterize optically opaque materials for controlling light delivery or performing imaging.
Direct Speech Translation for Automatic Subtitling
Automatic subtitling is the task of automatically translating the speech of audiovisual content into short pieces of timed text, i. e. subtitles and their corresponding timestamps.
Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora
Speech translation for subtitling (SubST) is the task of automatically translating speech data into well-formed subtitles by inserting subtitle breaks compliant to specific displaying guidelines.
Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
Simultaneous speech translation (SimulST) systems aim at generating their output with the lowest possible latency, which is normally computed in terms of Average Lagging (AL).
Who Are We Talking About? Handling Person Names in Speech Translation
Recent work has shown that systems for speech translation (ST) -- similarly to automatic speech recognition (ASR) -- poorly handle person names.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Efficient yet Competitive Speech Translation: FBK@IWSLT2022
The primary goal of this FBK's systems submission to the IWSLT 2022 offline and simultaneous speech translation tasks is to reduce model training costs without sacrificing translation quality.
Does Simultaneous Speech Translation need Simultaneous Models?
In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task.
Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation
Gender bias is largely recognized as a problematic phenomenon affecting language technologies, with recent studies underscoring that it might surface differently across languages.
Native state of natural proteins optimises local entropy
The differing ability of polypeptide conformations to act as the native state of proteins has long been rationalized in terms of differing kinetic accessibility or thermodynamic stability.
Visualization: the missing factor in Simultaneous Speech Translation
Simultaneous speech translation (SimulST) is the task in which output generation has to be performed on partial, incremental speech input.
Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translation
Automatic translation systems are known to struggle with rare words.
Speechformer: Reducing Information Loss in Direct Speech Translation
Transformer-based models have gained increasing popularity achieving state-of-the-art performance in many research fields including speech translation.
Ranked #1 on
Speech-to-Text Translation
on MuST-C EN->NL
Simultaneous Speech Translation for Live Subtitling: from Delay to Display
Experiments on three language pairs (en$\rightarrow$it, de, fr) show that scrolling lines is the only mode achieving an acceptable reading speed while keeping delay close to a 4-second threshold.
Between Flexibility and Consistency: Joint Generation of Captions and Subtitles
Speech translation (ST) has lately received growing interest for the generation of subtitles without the need for an intermediate source language transcription and timing (i. e. captions).
Dealing with training and test segmentation mismatch: FBK@IWSLT2021
Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora.
Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?
Five years after the first published proofs of concept, direct approaches to speech translation (ST) are now competing with traditional cascade solutions.
How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation
In light of this finding, we propose a combined approach that preserves BPE overall translation quality, while leveraging the higher ability of character-based segmentation to properly translate gender.
Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation
The audio segmentation mismatch between training data and those seen at run-time is a major problem in direct speech translation.
Gender Bias in Machine Translation
Machine translation (MT) technology has facilitated our daily tasks by providing accessible shortcuts for gathering, elaborating and communicating information.
Tutorial Proposal: End-to-End Speech Translation
Speech translation is the translation of speech in one language typically to text in another, traditionally accomplished through a combination of automatic speech recognition and machine translation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Self-Learning for Zero Shot Neural Machine Translation
Neural Machine Translation (NMT) approaches employing monolingual data are showing steady improvements in resource rich conditions.
CTC-based Compression for Direct Speech Translation
Previous studies demonstrated that a dynamic phone-informed compression of the input audio is beneficial for speech translation (ST).
The Multilingual TEDx Corpus for Speech Recognition and Translation
We present the Multilingual TEDx corpus, built to support speech recognition (ASR) and speech translation (ST) research across many non-English source languages.
On Knowledge Distillation for Direct Speech Translation
Direct speech translation (ST) has shown to be a complex task requiring knowledge transfer from its sub-tasks: automatic speech recognition (ASR) and machine translation (MT).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Breeding Gender-aware Direct Speech Translation Systems
In particular, by translating speech audio data without intermediate transcription, direct ST models are able to leverage and preserve essential information present in the input (e. g. speaker's vocal characteristics) that is otherwise lost in the cascade framework.
The Two Shades of Dubbing in Neural Machine Translation
Dubbing has two shades; synchronisation constraints are applied only when the actor{'}s mouth is visible on screen, while the translation is unconstrained for off-screen dubbing.
Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures
We analyze the connection between minimizers with good generalizing properties and high local entropy regions of a threshold-linear classifier in Gaussian mixtures with the mean squared error loss function.
On Target Segmentation for Direct Speech Translation
Then, subword-level segmentation became the state of the art in neural machine translation as it produces shorter sequences that reduce the training time, while being superior to word-level models.
Contextualized Translation of Automatically Segmented Speech
We show that our context-aware solution is more robust to VAD-segmented input, outperforming a strong base model and the fine-tuning on different VAD segmentations of an English-German test set by up to 4. 25 BLEU points.
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN
The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation.
Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus
Translating from languages without productive grammatical gender like English into gender-marked languages is a well-known difficulty for machines.
End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020
The test talks are provided in two versions: one contains the data already segmented with automatic tools and the other is the raw data without any segmentation.
Is 42 the Answer to Everything in Subtitling-oriented Speech Translation?
Subtitling is becoming increasingly important for disseminating information, given the enormous amounts of audiovisual content becoming available daily.
Low Resource Neural Machine Translation: A Benchmark for Five African Languages
Recent advents in Neural Machine Translation (NMT) have shown improvements in low-resource language (LRL) translation tasks.
MuST-Cinema: a Speech-to-Subtitles corpus
Growing needs in localising audiovisual content in multiple languages through subtitles call for the development of automatic solutions for human subtitling.
Adapting Multilingual Neural Machine Translation to Unseen Languages
In order to improve NMT for LRL, we employ perplexity to select HRL data that are most similar to the LRL on the basis of language distance.
Instance-Based Model Adaptation For Direct Speech Translation
Despite recent technology advancements, the effectiveness of neural approaches to end-to-end speech-to-text translation is still limited by the paucity of publicly available training corpora.
One-To-Many Multilingual End-to-end Speech Translation
Multilingual solutions are widely studied in MT and usually rely on ``\textit{target forcing}'', in which multilingual parallel data are combined to train a single model by prepending to the input sequences a language token that specifies the target language.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Machine Translation for Machines: the Sentiment Classification Use Case
Towards this objective, we present a reinforcement learning technique based on a new candidate sampling strategy, which exploits the results obtained on the downstream task as weak feedback.
Natural representation of composite data with replicated autoencoders
Generative processes in biology and other fields often produce data that can be regarded as resulting from a composition of basic features.
Multilingual Neural Machine Translation for Zero-Resource Languages
In recent years, Neural Machine Translation (NMT) has been shown to be more effective than phrase-based statistical methods, thus quickly becoming the state of the art in machine translation (MT).
Effort-Aware Neural Automatic Post-Editing
For this purpose, following the common approach in multilingual NMT, we prepend a special token to the beginning of both the source text and the MT output indicating the required amount of post-editing.
Findings of the WMT 2019 Shared Task on Automatic Post-Editing
Seven teams participated in the English-German task, with a total of 18 submitted runs.
Neural Text Simplification in Low-Resource Conditions Using Weak Supervision
Inspired by the machine translation field, in which synthetic parallel pairs generated from monolingual data yield significant improvements to neural models, in this paper we exploit large amounts of heterogeneous data to automatically select simple sentences, which are then used to create synthetic simplification pairs.
MuST-C: a Multilingual Speech Translation Corpus
Current research on spoken language translation (SLT) has to confront with the scarcity of sizeable and publicly available training corpora.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Improving Zero-Shot Translation of Low-Resource Languages
Recent work on multilingual neural machine translation reported competitive performance with respect to bilingual models and surprisingly good performance even on (zeroshot) translation directions not observed at training time.
Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary
Our approach allows to extend an initial model for a given language pair to cover new languages by adapting its vocabulary as long as new data become available (i. e., introducing new vocabulary items if they are not included in the initial model).
Generating E-Commerce Product Titles and Predicting their Quality
The setting requires the generation process to be fast and the generated title to be both human-readable and concise.
Fine-tuning on Clean Data for End-to-End Speech Translation: FBK @ IWSLT 2018
This paper describes FBK's submission to the end-to-end English-German speech translation task at IWSLT 2018.
Findings of the WMT 2018 Shared Task on Automatic Post-Editing
In the former subtask, characterized by original translations of lower quality, top results achieved impressive improvements, up to -6. 24 TER and +9. 53 BLEU points over the baseline {``}\textit{do-nothing}{''} system.
Multi-source transformer with combined losses for automatic post editing
In the first subtask, our system improves over the baseline up to -5. 3 TER and +8. 23 BLEU points ranking second out of 11 submitted runs.
eSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing
eSCAPE consists of millions of entries in which the MT element of the training triplets has been obtained by translating the source side of publicly-available parallel corpora, and using the target side as an artificial human post-edit.
Linguistically Motivated Vocabulary Reduction for Neural Machine Translation from Turkish to English
In this paper, we propose a new vocabulary reduction method for NMT, which can reduce the vocabulary of a given input corpus at any rate while also considering the morphological properties of the language.
Automatic Quality Estimation for ASR System Combination
In this paper we propose a novel variant of ROVER that takes advantage of ASR quality estimation (QE) for ranking the transcriptions at "segment level" instead of: i) relying on confidence scores, or ii) feeding ROVER with randomly ordered hypotheses.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Online Automatic Post-editing for MT in a Multi-Domain Translation Environment
Automatic post-editing (APE) for machine translation (MT) aims to fix recurrent errors made by the MT decoder by learning from correction examples.
Neural vs. Phrase-Based Machine Translation in a Multi-Domain Scenario
State-of-the-art neural machine translation (NMT) systems are generally trained on specific domains by carefully selecting the training sets and applying proper domain adaptation techniques.
DNN adaptation by automatic quality estimation of ASR hypotheses
Our hypothesis is that significant improvements can be achieved by: i)automatically transcribing the evaluation data we are currently trying to recognise, and ii) selecting from it a subset of "good quality" instances based on the word error rate (WER) scores predicted by a QE component.
Automatic Annotation of Machine Translation Datasets with Binary Quality Judgements
To overcome these issues, we present an automatic method for the annotation of (source, target) pairs with binary judgements that reflect an empirical, and easily interpretable notion of quality.
Chinese Whispers: Cooperative Paraphrase Acquisition
We present a framework for the acquisition of sentential paraphrases based on crowdsourcing.
