Search Results for author:
Found 24 papers, 4 papers with code
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
This article introduces AnCoGen, a novel method that leverages a masked autoencoder to unify the analysis, control, and generation of speech signals within a single model.
Fill in the Gap! Combining Self-supervised Representation Learning with Neural Audio Synthesis for Speech Inpainting
In the present study, we investigate the use of a speech SSL model for speech inpainting, that is reconstructing a missing portion of a speech signal from its surrounding context, i. e., fulfilling a downstream task that is very similar to the pretext task.
Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation
In this paper, we propose a latent-variable generative model called mixture of dynamical variational autoencoders (MixDVAE) to model the dynamics of a system composed of multiple moving sources.
Unsupervised speech enhancement with deep dynamical generative speech and noise models
This work builds on a previous work on unsupervised speech enhancement using a dynamical variational autoencoder (DVAE) as the clean speech model and non-negative matrix factorization (NMF) as the noise model.
A multimodal dynamical variational autoencoder for audiovisual speech representation learning
The latent space is structured to dissociate the latent dynamical factors that are shared between the modalities from those that are specific to each modality.
Speech Modeling with a Hierarchical Transformer Dynamical VAE
The dynamical variational autoencoders (DVAEs) are a family of latent-variable deep generative models that extends the VAE to model a sequence of observed data and a corresponding sequence of latent vectors.
BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model
We collect a corpus of utterances containing contrastive focus and we evaluate the accuracy of a BERT model, finetuned to predict quantized acoustic prominence features, on these samples.
Learning and controlling the source-filter representation of speech with a variational autoencoder
Using only a few seconds of labeled speech signals generated with an artificial speech synthesizer, we propose a method to identify the latent subspaces encoding $f_0$ and the first three formant frequencies, we show that these subspaces are orthogonal, and based on this orthogonality, we develop a method to accurately and independently control the source-filter speech factors within the latent subspaces.
Repeat after me: Self-supervised learning of acoustic-to-articulatory mapping by vocal imitation
We propose a computational model of speech production combining a pre-trained neural articulatory synthesizer able to reproduce complex speech stimuli from a limited set of interpretable articulatory parameters, a DNN-based internal forward model predicting the sensory consequences of articulatory commands, and an internal inverse model based on a recurrent neural network recovering articulatory commands from the acoustic speech input.
Unsupervised Multiple-Object Tracking with a Dynamical Variational Autoencoder
In this paper, we present an unsupervised probabilistic model and associated estimation algorithm for multi-object tracking (MOT) based on a dynamical variational autoencoder (DVAE), called DVAE-UMOT.
A Survey of Sound Source Localization with Deep Learning Methods
This article is a survey on deep learning methods for single and multiple sound source localization.
Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders
We propose an unsupervised speech enhancement algorithm that combines a DVAE speech prior pre-trained on clean speech signals with a noise model based on nonnegative matrix factorization, and we derive a variational expectation-maximization (VEM) algorithm to perform speech enhancement.
Learning robust speech representation with an articulatory-regularized variational autoencoder
It is increasingly considered that human speech perception and production both rely on articulatory representations.
Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input
The prosody of a spoken word is determined by its surrounding context.
What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS
In this paper, we study the behavior of a neural sequence-to-sequence TTS system when used in an incremental mode, i. e. when generating speech output for token n, the system has access to n + k tokens from the text sequence.
Dynamical Variational Autoencoders: A Comprehensive Review
Recently, a series of papers have presented different extensions of the VAE to process sequential data, which model not only the latent space but also the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks or state-space models.
A Recurrent Variational Autoencoder for Speech Enhancement
This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE).
Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoders
Variational auto-encoders (VAEs) are deep generative latent variable models that can be used for learning the distribution of complex data.
Speech enhancement with variational autoencoders and alpha-stable distributions
This paper focuses on single-channel semi-supervised speech enhancement.
A variance modeling framework based on variational autoencoders for speech enhancement
In this paper we address the problem of enhancing speech signals in noisy mixtures using a source separation approach.
Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization
In this paper we address speaker-independent multichannel speech enhancement in unknown noisy environments.
Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers
We propose a variational inference model which amounts to approximate the joint distribution with a factorized distribution.
Autoencoders for music sound modeling: a comparison of linear, shallow, deep, recurrent and variational models
This study investigates the use of non-linear unsupervised dimensionality reduction techniques to compress a music dataset into a low-dimensional representation which can be used in turn for the synthesis of new sounds.
Audio and Speech Processing Sound
Co-Localization of Audio Sources in Images Using Binaural Features and Locally-Linear Regression
Indeed, we demonstrate that the method can be used for audio-visual fusion, namely to map speech signals onto images and hence to spatially align the audio and visual modalities, thus enabling to discriminate between speaking and non-speaking faces.
