Search Results for author:
Found 24 papers, 9 papers with code
ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems
Motivated by this, we introduce an open-source, user-friendly toolkit designed to build unified web interfaces for various cascaded and E2E spoken dialogue systems.
Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM
Recent efforts have extended textual LLMs to the speech domain.
ESPnet-SpeechLM: An Open Speech Language Model Toolkit
We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications.
OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models
We use OWLS to derive neural scaling laws, showing how final performance can be reliably predicted when scaling.
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech
Neural codecs have become crucial to recent speech and audio generation research.
Preference Alignment Improves Language Model-Based TTS
Recent advancements in text-to-speech (TTS) have shown that language model (LM)-based systems offer competitive performance to their counterparts.
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild
This paper introduces SpoofCeleb, a dataset designed for Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV), utilizing source data from real-world conditions and spoofing attacks generated by Text-To-Speech (TTS) systems also trained on the same real-world data.
Text-To-Speech Synthesis In The Wild
The recent literature nonetheless shows efforts to train TTS systems using data collected in the wild.
Towards Robust Speech Representation Learning for Thousands of Languages
We propose XEUS, a Cross-lingual Encoder for Universal Speech, trained on over 1 million hours of data across 4057 languages, extending the language coverage of SSL models 4-fold.
On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models
The Open Whisper-style Speech Model (OWSM) series was introduced to achieve full transparency in building advanced speech-to-text (S2T) foundation models.
ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets
This paper presents ML-SUPERB~2. 0, which is a new benchmark for evaluating pre-trained SSL and supervised speech models across downstream models, fine-tuning setups, and efficient model adaptation approaches.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer
Recent studies have highlighted the importance of fully open foundation models.
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
Speech signals, typically sampled at rates in the tens of thousands per second, contain redundancies, evoking inefficiencies in sequence modeling.
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data
Pre-training speech models on large volumes of data has achieved remarkable success.
AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data
To address this issue, we introduce an automatic in-the-wild speech data preprocessing framework (AutoPrep) in this paper, which is designed to enhance speech quality, generate speaker labels, and produce transcriptions automatically.
Bayes Risk Transducer: Transducer with Controllable Alignment Prediction
While the vanilla transducer does not have a prior preference for any of the valid paths, this work intends to enforce the preferred paths and achieve controllable alignment prediction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence tasks
Besides predicting the target sequence, a side product of CTC is to predict the alignment, which is the most probable input-long sequence that specifies a hard aligning relationship between the input and target units.
LAE: Language-Aware Encoder for Monolingual and Multilingual ASR
Experiments conducted on Mandarin-English code-switched speech suggest that the proposed LAE is capable of discriminating different languages in frame-level and shows superior performance on both monolingual and multilingual ASR tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speaker-Aware Mixture of Mixtures Training for Weakly Supervised Speaker Extraction
Dominant researches adopt supervised training for speaker extraction, while the scarcity of ideally clean corpus and channel mismatch problem are rarely considered.
Integrating Lattice-Free MMI into End-to-End Speech Recognition
However, the effectiveness and efficiency of the MBR-based methods are compromised: the MBR criterion is only used in system training, which creates a mismatch between training and decoding; the on-the-fly decoding process in MBR-based methods results in the need for pre-trained models and slow training speeds.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model
Then, the LM score of the hypothesis is obtained by intersecting the generated lattice with an external word N-gram LM.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMI
Recently, End-to-End (E2E) frameworks have achieved remarkable results on various Automatic Speech Recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Layer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency
Transformer-based self-supervised models are trained as feature extractors and have empowered many downstream speech tasks to achieve state-of-the-art performance.
A Random Gossip BMUF Process for Neural Language Modeling
One important challenge of training an NNLM is to leverage between scaling the learning process and handling big data.
