Search Results for author:
Found 32 papers, 8 papers with code
Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset
Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity.
Generative Adversarial Network
Speech Emotion Recognition
+2
Emotional Voice Conversion: Theory, Databases and ESD
In this paper, we first provide a review of the state-of-the-art emotional voice conversion research, and the existing emotional speech databases.
Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data
Many studies require parallel speech data between different emotional patterns, which is not practical in real life.
Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training
In stage 2, we perform emotion training with a limited amount of emotional speech data, to learn how to disentangle emotional style and linguistic information from the speech.
Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion
We consider that there is a common code between speakers for emotional expression in a spoken language, therefore, a speaker-independent mapping between emotional states is possible.
StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis
The emotion strength of synthesized speech can be controlled flexibly using a strength descriptor, which is obtained by an emotion attribute ranking function.
Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning
In this paper, we propose a data-driven deep learning model, i. e. StrengthNet, to improve the generalization of emotion strength assessment for seen and unseen speech.
Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder
Accent plays a significant role in speech communication, influencing understanding capabilities and also conveying a person's identity.
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Our proposed approach significantly improved the intelligibility (in CER), the MOS, and discrimination ABX scores compared to the official ZeroSpeech 2019 baseline or even the topline.
Teacher-Student Training for Robust Tacotron-based TTS
We first train a Tacotron2-based TTS model by always providing natural speech frames to the decoder, that serves as a teacher model.
WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss
To address this problem, we propose a new training scheme for Tacotron-based TTS, referred to as WaveTTS, that has 2 loss functions: 1) time-domain loss, denoted as the waveform loss, that measures the distortion between the natural and generated waveform; and 2) frequency-domain loss, that measures the Mel-scale acoustic feature loss between the natural and generated acoustic features.
VAW-GAN for Singing Voice Conversion with Non-parallel Training Data
We train an encoder to disentangle singer identity and singing prosody (F0 contour) from phonetic content.
Spectrum and Prosody Conversion for Cross-lingual Voice Conversion with CycleGAN
It relies on non-parallel training data from two different languages, hence, is more challenging than mono-lingual voice conversion.
Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS
We propose a multi-task learning scheme for Tacotron training, that optimizes the system to predict both Mel spectrum and phrase breaks.
GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis
Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways.
VAW-GAN for Disentanglement and Recomposition of Emotional Elements in Speech
Emotional voice conversion (EVC) aims to convert the emotion of speech from one state to another while preserving the linguistic content and speaker identity.
Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability
To our best knowledge, this is the first study of reinforcement learning in emotional text-to-speech synthesis.
Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer
Traditional voice conversion(VC) has been focused on speaker identity conversion for speech with a neutral expression.
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
The proposed VisualTTS adopts two novel mechanisms that are 1) textual-visual attention, and 2) visual fusion strategy during acoustic decoding, which both contribute to forming accurate alignment between the input text content and lip motion in input lip sequence.
DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding
At the same time, as the deep neural analyzer is learnable, it is expected to be more accurate for signal reconstruction and manipulation, and generalizable from speech to singing.
Disentanglement of Emotional Style and Speaker Identity for Expressive Voice Conversion
Expressive voice conversion performs identity conversion for emotional speakers by jointly converting speaker identity and emotional style.
Emotion Intensity and its Control for Emotional Voice Conversion
As desired, the proposed network controls the fine-grained emotion intensity in the output speech.
Speech Synthesis with Mixed Emotions
We then incorporate our formulation into a sequence-to-sequence emotional text-to-speech framework.
Controllable Accented Text-to-Speech Synthesis
Accented TTS synthesis is challenging as L2 is different from L1 in both in terms of phonetic rendering and prosody pattern.
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models
In this work, we seek to answer the question: what are the characteristics of selected sparse techniques on the performance and model complexity?
Mixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion
To achieve this, we propose a novel EVC framework, Mixed-EVC, which only leverages discrete emotion training labels.
SNIPER Training: Variable Sparsity Rate Training For Text-To-Speech
Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary.
Versatile Audio-Visual Learning for Handling Single and Multi Modalities in Emotion Regression and Classification Tasks
This study proposes a \emph{versatile audio-visual learning} (VAVL) framework for handling unimodal and multimodal systems for emotion regression and emotion classification tasks.
Ranked #1 on
Video Emotion Recognition
on CREMA-D
Enhancing Speech Emotion Recognition Through Differentiable Architecture Search
In contrast to previous studies, we refrain from imposing constraints on the order of the layers for the CNN within the DARTS cell; instead, we allow DARTS to determine the optimal layer order autonomously.
Ranked #5 on
Speech Emotion Recognition
on IEMOCAP
(UA metric)
High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Units
The goal of Automatic Voice Over (AVO) is to generate speech in sync with a silent video given its text script.
Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition
In order to leverage this information, we introduce a novel contrastive pretraining approach applied to emotion-unlabeled data for speech emotion recognition.
emoDARTS: Joint Optimisation of CNN & Sequential Neural Network Architectures for Superior Speech Emotion Recognition
This study presents emoDARTS, a DARTS-optimised joint CNN and Sequential Neural Network (SeqNN: LSTM, RNN) architecture that enhances SER performance.
Ranked #1 on
Speech Emotion Recognition
on MSP-IMPROV
