Search Results for author:
Found 23 papers, 3 papers with code
The IWSLT 2019 Evaluation Campaign
The IWSLT 2019 evaluation campaign featured three tasks: speech translation of (i) TED talks and (ii) How2 instructional videos from English into German and Portuguese, and (iii) text translation of TED talks from English into Czech.
CMU’s Machine Translation System for IWSLT 2019
In Neural Machine Translation (NMT) the usage of sub-words and characters as source and target units offers a simple and flexible solution for translation of rare and unseen words.
Transforming LLMs into Cross-modal and Cross-lingual Retrieval Systems
Our multi-modal LLM-based retrieval system is capable of matching speech and text in 102 languages despite only training on 21 languages.
Layer-Wise Analysis of Self-Supervised Acoustic Word Embeddings: A Study on Speech Emotion Recognition
Through a comparative experiment and a layer-wise accuracy analysis on two distinct corpora, IEMOCAP and ESD, we explore differences between AWEs and raw self-supervised representations, as well as the proper utilization of AWEs alone and in combination with word embeddings.
Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Pooling
Acoustic word embeddings are typically created by training a pooling function using pairs of word-like units.
The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR
Although the great many advances in English automatic speech recognition (ASR) over the past decades, results are usually reported based on test datasets which fail to represent the diversity of English as spoken today around the globe.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models
Given the strong results of self-supervised models on various tasks, there have been surprisingly few studies exploring self-supervised representations for acoustic word embeddings (AWE), fixed-dimensional vectors representing variable-length spoken word segments.
Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Training
In this paper, we present a controlled study to better understand the effect of such factors on the performance of pre-trained representations on automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On the Difficulty of Segmenting Words with Attention
Word segmentation, the problem of finding word boundaries in speech, is of interest for a range of tasks.
Talk, Don't Write: A Study of Direct Speech-Based Image Retrieval
Speech-based image retrieval has been studied as a proxy for joint representation learning, usually without emphasis on retrieval itself.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multimodal Speech Recognition with Unstructured Audio Masking
Our experiments on the Flickr 8K Audio Captions Corpus show that multimodal ASR can generalize to recover different types of masked words in this unstructured masking setting.
Fine-Grained Grounding for Multimodal Speech Recognition
In experiments on the Flickr8K Audio Captions Corpus, we find that our model improves over approaches that use global visual features, that the proposals enable the model to recover entities and other related words, such as adjectives, and that improvements are due to the model's ability to localize the correct proposals.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Looking Enhances Listening: Recovering Missing Speech Using Images
Speech is understood better by using visual context; for this reason, there have been many attempts to use images to adapt automatic speech recognition (ASR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multitask Learning For Different Subword Segmentations In Neural Machine Translation
In Neural Machine Translation (NMT) the usage of subwords and characters as source and target units offers a simple and flexible solution for translation of rare and unseen words.
Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions
Multimodal learning allows us to leverage information from multiple sources (visual, acoustic and text), similar to our experience of the real world.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multimodal Grounding for Sequence-to-Sequence Speech Recognition
Specifically, in our previous work, we propose a multistep visual adaptive training approach which improves the accuracy of an audio-based Automatic Speech Recognition (ASR) system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
How2: A Large-scale Dataset for Multimodal Language Understanding
In this paper, we introduce How2, a multimodal collection of instructional videos with English subtitles and crowdsourced Portuguese translations.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Hierarchical Multi Task Learning With CTC
Our model obtains 14. 0% Word Error Rate on the Eval2000 Switchboard subset without any decoder or language model, outperforming the current state-of-the-art on acoustic-to-word models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
End-to-End Multimodal Speech Recognition
Transcription or sub-titling of open-domain videos is still a challenging domain for Automatic Speech Recognition (ASR) due to the data's challenging acoustics, variable signal processing and the essentially unrestricted domain of the data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Sequence-based Multi-lingual Low Resource Speech Recognition
Techniques for multi-lingual and cross-lingual speech recognition can help in low resource scenarios, to bootstrap systems and enable analysis of new languages and domains.
Subword and Crossword Units for CTC Acoustic Models
This paper proposes a novel approach to create an unit set for CTC based speech recognition systems.
Comparison of Decoding Strategies for CTC Acoustic Models
The CTC loss function maps an input sequence of observable feature vectors to an output sequence of symbols.
Robust end-to-end deep audiovisual speech recognition
Speech is one of the most effective ways of communication among humans.
