Search Results for author:
Found 43 papers, 12 papers with code
DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech
Recent advancements in Text-to-Speech (TTS) systems have enabled the generation of natural and expressive speech from textual input.
Discrete Unit based Masking for Improving Disentanglement in Voice Conversion
Voice conversion (VC) aims to modify the speaker's identity while preserving the linguistic content.
SelectTTS: Synthesizing Anyone's Voice via Discrete Unit-Based Frame Selection
With SelectTTS, we show that frame selection from the target speaker's speech is a direct way to achieve generalization in unseen speakers with low model complexity.
PRESENT: Zero-Shot Text-to-Prosody Control
We attain 25. 3% hanzi CER and 13. 0% pinyin CER with the JETS model.
We Need Variations in Speech Synthesis: Sub-center Modelling for Speaker Embeddings
The proposed approach introduces variations in the speaker embedding while retaining the speaker recognition performance since model does not have to map all of the utterances of a speaker into a single class center.
Odyssey 2024 - Speech Emotion Recognition Challenge: Dataset, Baseline Framework, and Results
The Odyssey 2024 Speech Emotion Recognition (SER) Challenge aims to enhance innovation in recognizing emotions from spontaneous speech, moving beyond traditional datasets derived from acted scenarios.
Towards Naturalistic Voice Conversion: NaturalVoices Dataset with an Automatic Processing Pipeline
Voice conversion (VC) research traditionally depends on scripted or acted speech, which lacks the natural spontaneity of real-life conversations.
Style Mixture of Experts for Expressive Text-To-Speech Synthesis
The proposed method replaces the style encoder in a TTS framework with a Mixture of Experts (MoE) layer.
Accent Conversion in Text-To-Speech Using Multi-Level VAE and Adversarial Training
With rapid globalization, the need to build inclusive and representative speech technology cannot be overstated.
Exploring speech style spaces with language models: Emotional TTS without emotion labels
Many frameworks for emotional text-to-speech (E-TTS) rely on human-annotated emotion labels that are often inaccurate and difficult to obtain.
Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model
A major challenge of expressive VC lies in emotion prosody modeling.
emoDARTS: Joint Optimisation of CNN & Sequential Neural Network Architectures for Superior Speech Emotion Recognition
This study presents emoDARTS, a DARTS-optimised joint CNN and Sequential Neural Network (SeqNN: LSTM, RNN) architecture that enhances SER performance.
Ranked #1 on
Speech Emotion Recognition
on MSP-IMPROV
Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition
In order to leverage this information, we introduce a novel contrastive pretraining approach applied to emotion-unlabeled data for speech emotion recognition.
High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Units
The goal of Automatic Voice Over (AVO) is to generate speech in sync with a silent video given its text script.
Enhancing Speech Emotion Recognition Through Differentiable Architecture Search
In contrast to previous studies, we refrain from imposing constraints on the order of the layers for the CNN within the DARTS cell; instead, we allow DARTS to determine the optimal layer order autonomously.
Ranked #2 on
Speech Emotion Recognition
on IEMOCAP
(UA metric)
Versatile audio-visual learning for emotion recognition
This study proposes a versatile audio-visual learning (VAVL) framework for handling unimodal and multimodal systems for emotion regression or emotion classification tasks.
Ranked #1 on
Arousal Estimation
on MSP-IMPROV
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech
Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary.
Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder
Accent plays a significant role in speech communication, influencing one's capability to understand as well as conveying a person's identity.
Mixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion
To achieve this, we propose a novel EVC framework, Mixed-EVC, which only leverages discrete emotion training labels.
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models
In this work, we seek to answer the question: what are the characteristics of selected sparse techniques on the performance and model complexity?
Controllable Accented Text-to-Speech Synthesis
Accented TTS synthesis is challenging as L2 is different from L1 in both in terms of phonetic rendering and prosody pattern.
Speech Synthesis with Mixed Emotions
We then incorporate our formulation into a sequence-to-sequence emotional text-to-speech framework.
Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning
In this paper, we propose a data-driven deep learning model, i. e. StrengthNet, to improve the generalization of emotion strength assessment for seen and unseen speech.
Emotion Intensity and its Control for Emotional Voice Conversion
As desired, the proposed network controls the fine-grained emotion intensity in the output speech.
Disentanglement of Emotional Style and Speaker Identity for Expressive Voice Conversion
Expressive voice conversion performs identity conversion for emotional speakers by jointly converting speaker identity and emotional style.
DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding
At the same time, as the deep neural analyzer is learnable, it is expected to be more accurate for signal reconstruction and manipulation, and generalizable from speech to singing.
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
The proposed VisualTTS adopts two novel mechanisms that are 1) textual-visual attention, and 2) visual fusion strategy during acoustic decoding, which both contribute to forming accurate alignment between the input text content and lip motion in input lip sequence.
StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis
The emotion strength of synthesized speech can be controlled flexibly using a strength descriptor, which is obtained by an emotion attribute ranking function.
Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer
Traditional voice conversion(VC) has been focused on speaker identity conversion for speech with a neutral expression.
Emotional Voice Conversion: Theory, Databases and ESD
In this paper, we first provide a review of the state-of-the-art emotional voice conversion research, and the existing emotional speech databases.
Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability
To our best knowledge, this is the first study of reinforcement learning in emotional text-to-speech synthesis.
Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training
In stage 2, we perform emotion training with a limited amount of emotional speech data, to learn how to disentangle emotional style and linguistic information from the speech.
VAW-GAN for Disentanglement and Recomposition of Emotional Elements in Speech
Emotional voice conversion (EVC) aims to convert the emotion of speech from one state to another while preserving the linguistic content and speaker identity.
Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset
Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity.
GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis
Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways.
Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS
We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks.
Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN
It relies on non-parallel training data from two different languages, hence, is more challenging than mono-lingual voice conversion.
VAW-GAN for Singing Voice Conversion with Non-parallel Training Data
We train an encoder to disentangle singer identity and singing prosody (F0 contour) from phonetic content.
Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion
We consider that there is a common code between speakers for emotional expression in a spoken language, therefore, a speaker-independent mapping between emotional states is possible.
WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss
To address this problem, we propose a new training scheme for Tacotron-based TTS, referred to as WaveTTS, that has 2 loss functions: 1) time-domain loss, denoted as the waveform loss, that measures the distortion between the natural and generated waveform; and 2) frequency-domain loss, that measures the Mel-scale acoustic feature loss between the natural and generated acoustic features.
Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data
Many studies require parallel speech data between different emotional patterns, which is not practical in real life.
Teacher-Student Training for Robust Tacotron-based TTS
We first train a Tacotron2-based TTS model by always providing natural speech frames to the decoder, that serves as a teacher model.
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Our proposed approach significantly improved the intelligibility (in CER), the MOS, and discrimination ABX scores compared to the official ZeroSpeech 2019 baseline or even the topline.
