Search Results for author:
Found 12 papers, 1 papers with code
SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
Unlike traditional modularised conversational AI systems, which separate speech recognition, understanding, and text-to-speech generation into distinct components, multimodal LLMs operate as single end-to-end models.
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
The results demonstrate that auditory LLMs achieve competitive performance compared to state-of-the-art task-specific small models in predicting MOS and SIM, while also delivering promising results in A/B testing and natural language descriptions.
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
We also conduct a comprehensive evaluation using objective evaluation methods (e. g. BLEU and ROUGE), subjective evaluations and LLM-based metrics for the generated responses.
CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing
To bridge the gap between modalities, CoAVT employs a query encoder, which contains a set of learnable query embeddings, and extracts the most informative audiovisual features of the corresponding text.
Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring
Specifically, we first pre-train the model using a reconstruction loss function, by masking phones and their durations jointly on a large amount of unlabeled speech and text prompts.
Leveraging phone-level linguistic-acoustic similarity for utterance-level pronunciation scoring
Recent studies on pronunciation scoring have explored the effect of introducing phone embeddings as reference pronunciation, but mostly in an implicit manner, i. e., addition or concatenation of reference phone embedding and actual pronunciation of the target phone as the phone-level pronunciation quality representation.
An ASR-free Fluency Scoring Approach with Self-Supervised Learning
A typical fluency scoring system generally relies on an automatic speech recognition (ASR) system to obtain time stamps in input speech for either the subsequent calculation of fluency-related features or directly modeling speech fluency with an end-to-end approach.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
TTS-Guided Training for Accent Conversion Without Parallel Data
Specifically, a text-to-speech (TTS) system is first pretrained with target-accented speech data.
Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information
Moreover, we utilize multi-source information (e. g., MFCC and deep features) to further improve the scoring system performance.
The Multi-speaker Multi-style Voice Cloning Challenge 2021
The challenge consists of two tracks, namely few-shot track and one-shot track, where the participants are required to clone multiple target voices with 100 and 5 samples respectively.
Spoofing detection under noisy conditions: a preliminary investigation and an initial database
To simulate the real-life scenarios, we perform a preliminary investigation of spoofing detection under additive noisy conditions, and also describe an initial database for this task.
A Waveform Representation Framework for High-quality Statistical Parametric Speech Synthesis
State-of-the-art statistical parametric speech synthesis (SPSS) generally uses a vocoder to represent speech signals and parameterize them into features for subsequent modeling.
