Search Results for author:
Found 22 papers, 12 papers with code
InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generation
This framework enables the controllable generation of high-fidelity long-form music at a higher sampling rate from both text and audio prompts.
SyncSpeech: Low-Latency and Efficient Dual-Stream Text-to-Speech based on Temporal Masked Transformer
This paper presents a dual-stream text-to-speech (TTS) model, SyncSpeech, capable of receiving streaming text input from upstream models while simultaneously generating streaming speech, facilitating seamless interaction with large language models.
Unispeaker: A Unified Approach for Multimodality-driven Speaker Generation
Recent advancements in personalized speech generation have brought synthetic speech increasingly close to the realism of target speakers' recordings, yet multimodal speaker generation remains on the rise.
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Previous models for voice interactions are categorized as native and aligned.
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode.
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
However, achieving low latency and natural interactions in full-duplex dialogue systems remains a significant challenge, especially considering human conversation dynamics such as interruptions, backchannels, and overlapping speech.
Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap
While automatic speech recognition (ASR) systems have achieved remarkable performance with large-scale datasets, their efficacy remains inadequate in low-resource settings, encompassing dialects, accents, minority languages, and long-tail hotwords, domains with significant practical relevance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilities
Current methods of building LLMs with voice interaction capabilities rely heavily on explicit text autoregressive generation before or during speech response generation to maintain content quality, which unfortunately brings computational overhead and increases latency in multi-turn interactions.
CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens
Based on the tokens, we further propose a scalable zero-shot TTS synthesizer, CosyVoice, which consists of an LLM for text-to-token generation and a conditional flow matching model for token-to-speech synthesis.
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs).
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
We found that delicate designs are not necessary, while an embarrassingly simple composition of off-the-shelf speech encoder, LLM, and the only trainable linear projector is competent for the ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
Previous mainstream audio-and-text LLMs use discrete audio tokens to represent both input and output audio; however, they suffer from performance degradation on tasks such as automatic speech recognition, speech-to-text translation, and speech enhancement over models using continuous speech features.
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec
We also demonstrate that the pre-trained models are suitable for downstream tasks, including automatic speech recognition and personalized text-to-speech synthesis.
CASA-ASR: Context-Aware Speaker-Attributed ASR
In addition, a two-pass decoding strategy is further proposed to fully leverage the contextual modeling ability resulting in a better recognition performance.
FunASR: A Fundamental End-to-End Speech Recognition Toolkit
FunASR offers models trained on large-scale industrial corpora and the ability to deploy them in applications.
Ranked #1 on
Speech Recognition
on WenetSpeech
(using extra training data)
TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization
Recently, end-to-end neural diarization (EEND) is introduced and achieves promising results in speaker-overlapped scenarios.
Ranked #1 on
Speaker Diarization
on CALLHOME
A Comparative Study on Multichannel Speaker-Attributed Automatic Speech Recognition in Multi-party Meetings
Speaker-attributed automatic speech recognition (SA-ASR) in multi-party meeting scenarios is one of the most valuable and challenging ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
Therefore, we propose the second approach, WD-SOT, to address alignment errors by introducing a word-level diarization model, which can get rid of such timestamp alignment dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
Through this formulation, we propose the speaker embedding-aware neural diarization (SEND) framework, where a speech encoder, a speaker encoder, two similarity scorers, and a post-processing network are jointly optimized to predict the encoded labels according to the similarities between speech features and speaker embeddings.
Ranked #1 on
Speaker Diarization
on AliMeeting
Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information
In this paper, we reformulate this task as a single-label prediction problem by encoding the multi-speaker labels with power set.
A Monaural Speech Enhancement Method for Robust Small-Footprint Keyword Spotting
To improve the robustness, a speech enhancement front-end is involved.
Acoustic Scene Classification by Implicitly Identifying Distinct Sound Events
In this paper, we propose a new strategy for acoustic scene classification (ASC) , namely recognizing acoustic scenes through identifying distinct sound events.
