Search Results for author:
Found 15 papers, 9 papers with code
Pretrained Speech Encoders and Efficient Fine-tuning Methods for Speech Translation: UPC at IWSLT 2022
We further investigate the suitability of different speech encoders (wav2vec 2. 0, HuBERT) for our models and the impact of knowledge distillation from the Machine Translation model that we use for the decoder (mBART).
Pushing the Limits of Zero-shot End-to-End Speech Translation
The speech encoder seamlessly integrates with the MT model at inference, enabling direct translation from speech to text, across all languages supported by the MT model.
SpeechAlign: a Framework for Speech Translation Alignment Evaluation
In doing so, we contribute to the ongoing research progress within the fields of Speech-to-Speech and Speech-to-Text translation.
Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT23
Our Speech Translation systems utilize foundation models for speech (wav2vec 2. 0) and text (mBART50).
Explaining How Transformers Use Context to Build Predictions
Language Generation Models produce words based on the previous context.
Sign Language Translation from Instructional Videos
We report a result of 8. 03 on the BLEU score, and publish the first open-source implementation of its kind to promote further advances.
Ranked #1 on
Sign Language Translation
on How2Sign
Efficient Speech Translation with Dynamic Latent Perceivers
Transformers have been the dominant architecture for Speech Translation in recent years, achieving significant improvements in translation quality.
Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer
In Neural Machine Translation (NMT), each token prediction is conditioned on the source sentence and the target prefix (what has been previously translated at a decoding step).
Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation
Different approaches have been proposed to overcome these problems, such as the use of efficient attention mechanisms.
On the Locality of Attention in Direct Speech Translation
Transformers have achieved state-of-the-art results across multiple NLP tasks.
Measuring the Mixing of Contextual Information in the Transformer
The Transformer architecture aggregates input information through the self-attention mechanism, but there is no clear understanding of how this information is mixed across the entire model.
SHAS: Approaching optimal Segmentation for End-to-End Speech Translation
Speech translation datasets provide manual segmentations of the audios, which are not available in real-world scenarios, and existing segmentation methods usually significantly reduce translation quality at inference time.
Efficient Transformer for Direct Speech Translation
When working with speech, we must face a problem: the sequence length of an audio input is not suitable for the Transformer.
End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021
Our submission also uses a custom segmentation algorithm that employs pre-trained Wav2Vec 2. 0 for identifying periods of untranscribable text and can bring improvements of 2. 5 to 3 BLEU score on the IWSLT 2019 test set, as compared to the result with the given segmentation.
Ranked #2 on
Speech-to-Text Translation
on MuST-C EN->DE
(using extra training data)
Evaluating Gender Bias in Speech Translation
WinoST is the speech version of WinoMT which is a MT challenge set and both follow an evaluation protocol to measure gender accuracy.
