Search Results for author:
Found 27 papers, 7 papers with code
TS3-Codec: Transformer-Based Simple Streaming Single Codec
Neural audio codecs (NACs) have garnered significant attention as key technologies for audio compression as well as audio representation for speech language models.
Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
This paper introduces EmoCtrl-TTS, an emotion-controllable zero-shot TTS that can generate highly emotional speech with NVs for any speaker.
Target conversation extraction: Source separation using turn-taking dynamics
Extracting the speech of participants in a conversation amidst interfering speakers and noise presents a challenging problem.
Knowledge boosting during low-latency inference
Models for low-latency, streaming applications could benefit from the knowledge capacity of larger models, but edge devices cannot run these models due to resource constraints.
E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS
This paper introduces Embarrassingly Easy Text-to-Speech (E2 TTS), a fully non-autoregressive zero-shot text-to-speech system that offers human-level naturalness and state-of-the-art speaker similarity and intelligibility.
An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
Recently, zero-shot text-to-speech (TTS) systems, capable of synthesizing any speaker's voice from a short audio prompt, have made rapid advancements.
Total-Duration-Aware Duration Modeling for Text-to-Speech Systems
We also show that the proposed MaskGIT-based model can generate phoneme durations with higher quality and diversity compared to its regression or flow-matching counterparts.
Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like
In this work, we propose ELaTE, a zero-shot TTS that can generate natural laughing speech of any speaker based on a short audio prompt with precise control of laughter timing and expression.
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Recent advancements in generative speech models based on audio-text prompts have enabled remarkable innovations like high-quality zero-shot text-to-speech.
Real-Time Audio-Visual End-to-End Speech Enhancement
In this paper, we propose a low-latency real-time audio-visual end-to-end enhancement (AV-E3Net) model based on the recently proposed end-to-end enhancement network (E3Net).
Speech separation with large-scale self-supervised learning
Compared with a supervised baseline and the WavLM-based SS model using feature embeddings obtained with the previously released 94K hours trained WavLM, our proposed model obtains 15. 9% and 11. 2% of relative word error rate (WER) reductions, respectively, for a simulated far-field speech mixture test set.
Breaking the trade-off in personalized speech enhancement with cross-task knowledge distillation
This prevents the PSE model from being too aggressive while still allowing the model to learn to suppress the input speech when it is likely to be spoken by interfering speakers.
Real-Time Joint Personalized Speech Enhancement and Acoustic Echo Cancellation
Personalized speech enhancement (PSE) is a real-time SE approach utilizing a speaker embedding of a target person to remove background noise, reverberation, and interfering voices.
Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation
In this paper, we propose a three-stage training scheme for the CSS model that can leverage both supervised data and extra large-scale unsupervised real-world conversational data.
Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation
Our results show that E3Net provides better speech and transcription quality with a lower target speaker over-suppression (TSOS) rate than the baseline model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ICASSP 2022 Deep Noise Suppression Challenge
We open-source datasets and test sets for researchers to train their deep noise suppression models, as well as a subjective evaluation framework based on ITU-T P. 835 to rate and rank-order the challenge entries.
Sequence-level self-learning with multiple hypotheses
For untranscribed speech data, the hypothesis from an ASR system must be used as a label.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Multi-talker conversational speech processing has drawn many interests for various applications such as meeting transcription.
One model to enhance them all: array geometry agnostic multi-channel personalized speech enhancement
Experimental results show that the proposed geometry agnostic model outperforms the model trained on a specific microphone array geometry in both speech quality and automatic speech recognition accuracy.
Personalized Speech Enhancement: New Models and Comprehensive Evaluation
Our results show that the proposed models can yield better speech recognition accuracy, speech intelligibility, and perceptual quality than the baseline models, and the multi-task training can alleviate the TSOS issue in addition to improving the speech recognition accuracy.
All-neural beamformer for continuous speech separation
Recently, the all deep learning MVDR (ADL-MVDR) model was proposed for neural beamforming and demonstrated superior performance in a target speech extraction task using pre-segmented input.
Dynamic Gradient Aggregation for Federated Domain Adaptation
The proposed scheme is based on a weighted gradient aggregation using two-step optimization to offer a flexible training pipeline.
Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement
Performance analysis is also carried out by changing the ASR model, the data used for the ASR-step, and the schedule of the two update steps.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition
Visual emotion expression plays an important role in audiovisual speech communication.
Federated Transfer Learning with Dynamic Gradient Aggregation
The target scenario is Acoustic Model training based on this platform.
Improving Readability for Automatic Speech Recognition Transcription
In this work, we propose a novel NLP task called ASR post-processing for readability (APR) that aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Generating Talking Face Landmarks from Speech
In this paper, we present a system that can generate landmark points of a talking face from an acoustic speech in real time.
