Search Results for author:
Found 20 papers, 11 papers with code
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Previous models for voice interactions are categorized as native and aligned.
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode.
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
However, achieving low latency and natural interactions in full-duplex dialogue systems remains a significant challenge, especially considering human conversation dynamics such as interruptions, backchannels, and overlapping speech.
Exploring Text-Queried Sound Event Detection with Audio Source Separation
To address this issue, we propose the text-queried SED (TQ-SED) framework.
Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts
To improve readability, we propose a Contextualized Spoken-to-Written conversion (CoS2W) task to address ASR and grammar errors and also transfer the informal text into the formal style with content preserved, utilizing contexts and auxiliary information.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multimodal Fusion and Coherence Modeling for Video Topic Segmentation
In this work, we improve supervised VTS by thoroughly exploring multimodal fusion and multimodal coherence modeling.
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs).
Skip-Layer Attention: Bridging Abstract and Detailed Dependencies in Transformers
The Transformer architecture has significantly advanced deep learning, particularly in natural language processing, by effectively managing long-range dependencies.
3D-Speaker-Toolkit: An Open-Source Toolkit for Multimodal Speaker Verification and Diarization
With 3D-Speaker-Toolkit, we establish a new benchmark for multimodal speaker analysis.
Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
We find that applying the conventional cross-entropy loss on input speech tokens does not consistently improve the ASR performance over the Loss Masking approach.
Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling
Our approach improve $F_1$ of old SOTA by 3. 42 (73. 74 -> 77. 16) and reduces $P_k$ by 1. 11 points (15. 0 -> 13. 89) on WIKI-727K and achieves an average relative reduction of 4. 3% on $P_k$ on WikiSection.
Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision
To mitigate this problem, we introduce a diversity regularization term to embeddings in SDPN.
Improving BERT with Hybrid Pooling Network and Drop Mask
Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks.
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings
Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e. g., BERT, without fine-tuning.
Meeting Action Item Detection with Regularized Context Modeling
We construct and release the first Chinese meeting corpus with manual action item annotations.
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG)
ICASSP2023 General Meeting Understanding and Generation Challenge (MUG) focuses on prompting a wide range of spoken language processing (SLP) research on meeting transcripts, as SLP applications are critical to improve users' efficiency in grasping important information in meetings.
MUG: A General Meeting Understanding and Generation Benchmark
To prompt SLP advancement, we establish a large-scale general Meeting Understanding and Generation Benchmark (MUG) to benchmark the performance of a wide range of SLP tasks, including topic segmentation, topic-level and session-level extractive summarization and topic title generation, keyphrase extraction, and action item detection.
Weighted Sampling for Masked Language Modeling
Experiments on the Semantic Textual Similarity benchmark (STS) show that WSBERT significantly improves sentence embeddings over BERT.
MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction
In this work, we propose a novel unsupervised embedding-based KPE approach, Masked Document Embedding Rank (MDERank), to address this problem by leveraging a mask strategy and ranking candidates by the similarity between embeddings of the source document and the masked document.
LCQMC:A Large-scale Chinese Question Matching Corpus
In this paper, we first use a search engine to collect large-scale question pairs related to high-frequency words from various domains, then filter irrelevant pairs by the Wasserstein distance, and finally recruit three annotators to manually check the left pairs.
