Search Results for author:
Found 62 papers, 32 papers with code
LiSTra Automatic Speech Translation: English to Lingala Case Study
In recent years there has been great interest in addressing the data scarcity of African languages and providing baseline models for different Natural Language Processing tasks (Orife et al., 2020).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
TransFusion: Transcribing Speech with Multinomial Diffusion
In this work we aim to see whether the benefits of diffusion models can also be realized for speech recognition.
Towards visually prompted keyword localisation for zero-resource spoken languages
We formalise this task and call it visually prompted keyword localisation (VPKL): given an image of a keyword, detect and predict where in an utterance the keyword occurs.
GAN You Hear Me? Reclaiming Unconditional Speech Synthesis from Diffusion Models
As in the StyleGAN family of image synthesis models, ASGAN maps sampled noise to a disentangled latent vector which is then mapped to a sequence of audio features so that signal aliasing is suppressed at every layer.
YFACC: A Yorùbá speech-image dataset for cross-lingual keyword localisation through visual grounding
We collect and release a new single-speaker dataset of audio captions for 6k Flickr images in Yor\`ub\'a -- a real low-resource language spoken in Nigeria.
A Temporal Extension of Latent Dirichlet Allocation for Unsupervised Acoustic Unit Discovery
In this paper, we present an extension to LDA that uses a Markov chain to model temporal information.
Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
This paper instead revisits an older approach to word segmentation: bottom-up phone-like unit discovery is performed first, and symbolic word segmentation is then performed on top of the discovered units (without influencing the lower level).
Keyword localisation in untranscribed speech using visually grounded speech models
Masked-based localisation gives some of the best reported localisation scores from a VGS model, with an accuracy of 57% when the system knows that a keyword occurs in an utterance and need to predict its location.
Voice Conversion Can Improve ASR in Very Low-Resource Settings
In this work we assess whether a VC system can be used cross-lingually to improve low-resource speech recognition.
Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel
The Speaker is equipped with a vocoder that maps symbols to a continuous waveform, this is passed over a lossy continuous channel, and the Listener needs to map the continuous signal to the concept.
A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion
Specifically, we compare discrete and soft speech units as input features.
Feature learning for efficient ASR-free keyword spotting in low-resource languages
We show that, using these features, the CNN-DTW keyword spotter performs almost as well as the DTW keyword spotter while outperforming a baseline CNN trained only on the keyword templates.
Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing
In this paper, we first show that the per-utterance mean of CPC features captures speaker information to a large extent.
Mava: a research framework for distributed multi-agent reinforcement learning
We provide experimental results for these implementations on a wide range of multi-agent environments and highlight the benefits of distributed system training.
Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language
Through finer-grained analysis, we show that training on even just a single related language gives the largest gain.
Attention-Based Keyword Localisation in Speech using Visual Grounding
Visually grounded speech models learn from images paired with spoken captions.
StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contexts
We specifically extend the recent StarGAN-VC model by conditioning it on a speaker embedding (from a potentially unseen speaker).
Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation
We consider how a recent contrastive learning loss can be used in both the purely unsupervised and multilingual transfer settings.
A phonetic model of non-native spoken word processing
We then test the model on a spoken word processing task, showing that phonology may not be necessary to explain some of the word processing effects observed in non-native speakers.
A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings
We compare frame-level features from contrastive predictive coding (CPC), autoregressive predictive coding and a CAE to conventional MFCCs.
Towards localisation of keywords in speech using weak supervision
Of the two forms of supervision, the visually trained model performs worse than the BoW-trained model.
Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks
We specifically constrain pretrained self-supervised vector-quantized (VQ) neural networks so that blocks of contiguous feature vectors are assigned to the same code, thereby giving a variable-rate segmentation of the speech into discrete units.
Direct multimodal few-shot learning of speech and images
We propose direct multimodal few-shot models that learn a shared embedding space of spoken words and images from only a few paired examples.
A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings
We propose a new unsupervised model for mapping a variable-duration speech segment to a fixed-dimensional representation.
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved.
Unsupervised vs. transfer learning for multimodal one-shot matching of speech and images
Here we compare transfer learning to unsupervised models trained on unlabelled in-domain data.
Evaluating computational models of infant phonetic learning across languages
In the first year of life, infants' speech perception becomes attuned to the sounds of their native language.
Improved acoustic word embeddings for zero-resource languages using multilingual transfer
We consider three multilingual recurrent neural network (RNN) models: a classifier trained on the joint vocabularies of all training languages; a Siamese RNN trained to discriminate between same and different words from multiple languages; and a correspondence autoencoder (CAE) RNN trained to reconstruct word pairs.
Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge
The idea is to learn a representation of speech by predicting future acoustic units.
Ranked #1 on
Acoustic Unit Discovery
on ZeroSpeech 2019 English
Analyzing autoencoder-based acoustic word embeddings
To better understand the applications of AWEs in various downstream tasks and in cognitive modeling, we need to analyze the representation spaces of AWEs.
Unsupervised feature learning for speech using correspondence and Siamese networks
Dynamic programming is then used to align the feature frames between each word pair, serving as weak top-down supervision for the two models.
Multilingual acoustic word embedding models for processing zero-resource languages
Acoustic word embeddings are fixed-dimensional representations of variable-length speech segments.
BINet: a binary inpainting network for deep patch-based image compression
We propose the Binary Inpainting Network (BINet), an autoencoder framework which incorporates binary inpainting to reinstate interdependencies between adjacent patches, for improved patch-based compression of still images.
Deep motion estimation for parallel inter-frame prediction in video compression
Standard video codecs rely on optical flow to guide inter-frame prediction: pixels from reference frames are moved via motion vectors to predict target video frames.
If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks
Our results therefore suggest that, in the shallow-to-moderate depth setting, critical initialisation provides zero performance gains when compared to off-critical initialisations and that searching for off-critical initialisations that might improve training speed or generalisation, is likely to be a fruitless endeavour.
On the expected behaviour of noise regularised deep neural networks as Gaussian processes
Recent work has established the equivalence between deep neural networks and Gaussian processes (GPs), resulting in so-called neural network Gaussian processes (NNGPs).
Cross-lingual topic prediction for speech using translations
Given a large amount of unannotated speech in a low-resource language, can we classify the speech utterances by topic?
On the Contributions of Visual and Textual Supervision in Low-Resource Semantic Speech Retrieval
Recent work has shown that speech paired with images can be used to learn semantically meaningful speech representations even without any textual supervision.
Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks
For our submission to the ZeroSpeech 2019 challenge, we apply discrete latent-variable neural networks to unlabelled speech and use the discovered units for speech synthesis.
Semantic query-by-example speech search using visual grounding
A number of recent studies have started to investigate how speech systems can be trained on untranscribed speech by leveraging accompanying images at training time.
Feature exploration for almost zero-resource ASR-free keyword spotting using a multilingual bottleneck extractor and correspondence autoencoders
We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting.
Multilingual and Unsupervised Subword Modeling for Zero-Resource Languages
Here we directly compare multiple methods, including some that use only target language speech data and some that use transcribed speech from other (non-target) languages, and we evaluate using two intrinsic measures as well as on a downstream unsupervised word segmentation and clustering task.
Multimodal One-Shot Learning of Speech and Images
Imagine a robot is shown new concepts visually together with spoken tags, e. g. "milk", "eggs", "butter".
Critical initialisation for deep signal propagation in noisy rectifier neural networks
Simulations and experiments on real-world data confirm that our proposed initialisation is able to stably propagate signals in deep networks, while using an initialisation disregarding noise fails to do so.
Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models
We investigate unsupervised models that can map a variable-duration speech segment to a fixed-dimensional representation.
Pre-training on high-resource speech recognition improves low-resource speech-to-text translation
Finally, we show that the approach improves performance on a true low-resource task: pre-training on a combination of English ASR and French ASR improves Mboshi-French ST, where only 4 hours of data are available, from 3. 5 to 7. 1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
ASR-free CNN-DTW keyword spotting using multilingual bottleneck features for almost zero-resource languages
We consider multilingual bottleneck features (BNFs) for nearly zero-resource keyword spotting.
Fast ASR-free and almost zero-resource keyword spotting using DTW and CNNs for humanitarian monitoring
While the resulting CNN keyword spotter cannot match the performance of the DTW-based system, it substantially outperforms a CNN classifier trained only on the keywords, improving the area under the ROC curve from 0. 54 to 0. 64.
Learning Dynamics of Linear Denoising Autoencoders
Here we develop theory for how noise influences learning in DAEs.
Visually grounded cross-lingual keyword spotting in speech
Recent work considered how images paired with speech can be used as supervision for building speech systems when transcriptions are not available.
Low-Resource Speech-to-Text Translation
We explore models trained on between 20 and 160 hours of data, and find that although models trained on less data have considerably lower BLEU scores, they can still predict words with relatively high precision and recall---around 50% for a model trained on 50 hours of data, versus around 60% for the full 160 hour model.
Semantic speech retrieval with a visually grounded model of untranscribed speech
We introduce a newly collected data set of human semantic relevance judgements and an associated task, semantic speech retrieval, where the goal is to search for spoken utterances that are semantically relevant to a given text query.
Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings
Query-by-example search often uses dynamic time warping (DTW) for comparing queries and proposed matching segments.
An embedded segmental K-means model for unsupervised segmentation and clustering of speech
Unsupervised segmentation and clustering of unlabelled speech are core problems in zero-resource speech processing.
Visually grounded learning of keyword prediction from untranscribed speech
In this setting of images paired with untranscribed spoken captions, we consider whether computer vision systems can be used to obtain textual labels for the speech.
Towards speech-to-text translation without speech recognition
We explore the problem of translating speech to text in low-resource scenarios where neither automatic speech recognition (ASR) nor machine translation (MT) are available, but we have training data in the form of audio paired with text translations.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Unsupervised neural and Bayesian models for zero-resource speech processing
Finally, we show that the clusters discovered by the segmental Bayesian model can be made less speaker- and gender-specific by using features from the cAE instead of traditional acoustic features.
Weakly supervised spoken term discovery using cross-lingual side information
Recent work on unsupervised term discovery (UTD) aims to identify and cluster repeated word-like units from audio alone.
A segmental framework for fully-unsupervised large-vocabulary speech recognition
We also show that the discovered clusters can be made less speaker- and gender-specific by using an unsupervised autoencoder-like feature extractor to learn better frame-level features (prior to embedding).
Unsupervised word segmentation and lexicon discovery using acoustic word embeddings
In settings where only unlabelled speech data is available, speech technology needs to be developed without transcriptions, pronunciation dictionaries, or language modelling text.
Deep convolutional acoustic word embeddings using word-pair side information
Recent studies have been revisiting whole words as the basic modelling unit in speech recognition and query applications, instead of phonetic units.
