Search Results for author:
Found 36 papers, 9 papers with code
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
CoVoMix is capable of first converting dialogue text into multiple streams of discrete tokens, with each token stream representing semantic information for individual talkers.
Improving Design of Input Condition Invariant Speech Enhancement
In this paper we propose novel architectures to improve the input condition invariant SE model so that performance in simulated conditions remains competitive while real condition degradation is much mitigated.
Prompt-driven Target Speech Diarization
We introduce a novel task named `target speech diarization', which seeks to determine `when target event occurred' within an audio signal.
One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
Various Large Language Models(LLMs) from the Generative Pretrained Transformer(GPT) family have achieved outstanding performances in a wide range of text generation tasks.
Toward Universal Speech Enhancement for Diverse Input Conditions
Currently, there is no universal SE approach that can effectively handle diverse input conditions with a single model.
Diffusion Conditional Expectation Model for Efficient and Robust Target Speech Extraction
Additionally, we introduce Regenerate-DCEM (R-DCEM) that can regenerate and optimize speech quality based on pre-processed speech from a discriminative model.
Leveraging In-the-Wild Data for Effective Self-Supervised Pretraining in Speaker Recognition
Current speaker recognition systems primarily rely on supervised approaches, constrained by the scale of labeled datasets.
InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models
We tested InstructME in instrument-editing, remixing, and multi-round editing.
Exploring the Integration of Speech Separation and Recognition with Self-Supervised Learning Representation
In detail, we explore multi-channel separation methods, mask-based beamforming and complex spectral mapping, as well as the best features to use in the ASR back-end model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Exploring Binary Classification Loss For Speaker Verification
The mismatch between close-set training and open-set testing usually leads to significant performance degradation for speaker verification task.
Adapting Multi-Lingual ASR Models for Handling Multiple Talkers
State-of-the-art large-scale universal speech models (USMs) show a decent automatic speech recognition (ASR) performance across multiple domains and languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Weakly-Supervised Speech Pre-training: A Case Study on Target Speech Recognition
Self-supervised learning (SSL) based speech pre-training has attracted much attention for its capability of extracting rich representations learned from massive unlabeled data.
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation
Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language.
Whisper-KDQ: A Lightweight Whisper via Guided Knowledge Distillation and Quantization for Efficient ASR
Due to the rapid development of computing hardware resources and the dramatic growth of data, pre-trained models in speech recognition, such as Whisper, have significantly improved the performance of speech recognition tasks.
Code-Switching Text Generation and Injection in Mandarin-English ASR
Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Target Sound Extraction with Variable Cross-modality Clues
Automatic target sound extraction (TSE) is a machine learning approach to mimic the human auditory perception capability of attending to a sound source of interest from a mixture of sources.
LongFNT: Long-form Speech Recognition with Factorized Neural Transducer
This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines
In the metric aspect, we design the new conversational DER (CDER) evaluation metric, which calculates the SD accuracy at the utterance level.
ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding
To showcase such integration, we performed experiments on carefully designed synthetic datasets for noisy-reverberant multi-channel ST and SLU tasks, which can be used as benchmark corpora for future research.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
End-to-End Multi-speaker ASR with Independent Vector Analysis
We develop an end-to-end system for multi-channel, multi-speaker automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SkiM: Skipping Memory LSTM for Low-Latency Real-Time Continuous Speech Separation
We adopt the time-domain speech separation method and the recently proposed Graph-PIT to build a super low-latency online speech separation model, which is very important for the real application.
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Multi-talker conversational speech processing has drawn many interests for various applications such as meeting transcription.
Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditions
The deep learning based time-domain models, e. g. Conv-TasNet, have shown great potential in both single-channel and multi-channel speech enhancement.
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks.
Dual-Path Modeling for Long Recording Speech Separation in Meetings
A transformer-based dual-path system is proposed, which integrates transform layers for global modeling.
End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontend
Recently, the end-to-end approach has been successfully applied to multi-speaker speech separation and recognition in both single-channel and multichannel conditions.
Data Augmentation for End-to-end Code-switching Speech Recognition
Training a code-switching end-to-end automatic speech recognition (ASR) model normally requires a large amount of data, while code-switching data is often limited.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Future Vector Enhanced LSTM Language Model for LVCSR
For the speech recognition rescoring, although the proposed LSTM LM obtains very slight gains, the new model seems obtain the great complementary with the conventional LSTM LM.
End-to-End Multi-speaker Speech Recognition with Transformer
Recently, fully recurrent neural network (RNN) based end-to-end models have been proven to be effective for multi-speaker speech recognition in both the single-channel and multi-channel scenarios.
MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
In this work, we propose a novel neural sequence-to-sequence (seq2seq) architecture, MIMO-Speech, which extends the original seq2seq to deal with multi-channel input and multi-channel output so that it can fully model multi-channel multi-speaker speech separation and recognition.
Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition
The proposed approach can achieve the state-of-the-art performance, with 25% ~ 30% equal error rate (EER) reduction on both tasks when compared to strong baselines using cross entropy loss with softmax, obtaining 2. 238% EER on VoxCeleb1 test set and 2. 761% EER on SITW core-core test set, respectively.
End-to-End Monaural Multi-speaker ASR System without Pretraining
The experiments demonstrate that the proposed methods can improve the performance of the end-to-end model in separating the overlapping speech and recognizing the separated streams.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Sequence Discriminative Training for Deep Learning based Acoustic Keyword Spotting
The few studies on sequence discriminative training for KWS are limited for fixed vocabulary or LVCSR based methods and have not been compared to the state-of-the-art deep learning based KWS approaches.
Single-Channel Multi-talker Speech Recognition with Permutation Invariant Training
Although great progresses have been made in automatic speech recognition (ASR), significant performance degradation is still observed when recognizing multi-talker mixed speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Recognizing Multi-talker Speech with Permutation Invariant Training
Our technique is based on permutation invariant training (PIT) for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Very Deep Convolutional Neural Networks for Robust Speech Recognition
On the Aurora 4 task, the very deep CNN achieves a WER of 8. 81%, further 7. 99% with auxiliary feature joint training, and 7. 09% with LSTM-RNN joint decoding.
