Search Results for author:
Found 26 papers, 13 papers with code
3D-Speaker-Toolkit: An Open Source Toolkit for Multi-modal Speaker Verification and Diarization
This paper introduces 3D-Speaker-Toolkit, an open source toolkit for multi-modal speaker verification and diarization.
Language-Codec: Reducing the Gaps Between Discrete Codec Representation and Speech Language Models
Furthermore, we also validate the efficiency of the Language-Codec on downstream speech language models.
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
We found that delicate designs are not necessary, while an embarrassingly simple composition of off-the-shelf speech encoder, LLM, and the only trainable linear projector is competent for the ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
We find that applying the conventional cross-entropy loss on input speech tokens does not consistently improve the ASR performance over the Loss Masking approach.
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
In this paper, we propose LauraGPT, a unified GPT model for audio recognition, understanding, and generation.
Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation
Speaker diarization has gained considerable attention within speech processing research community.
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec
We also demonstrate that the pre-trained models are suitable for downstream tasks, including automatic speech recognition and personalized text-to-speech synthesis.
Self-Distillation Network with Ensemble Prototypes: Learning Robust Speaker Representations without Supervision
It assigns representation of augmented views of utterances to the same prototypes as the representation of the original view, thereby enabling effective knowledge transfer between the views.
Improving BERT with Hybrid Pooling Network and Drop Mask
Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks.
3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement
Disentangling uncorrelated information in speech utterances is a crucial research topic within speech community.
Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization
In this paper, we propose methods to extract speaker-related information from semantic content in multi-party meetings, which, as we will show, can further benefit speaker diarization.
An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification
This paper proposes a novel architecture called Enhanced Res2Net (ERes2Net), which incorporates both local and global feature fusion techniques to improve the performance.
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings
Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e. g., BERT, without fine-tuning.
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect
Recently, binaural audio synthesis (BAS) has emerged as a promising research field for its applications in augmented and virtual realities.
Contextual Expressive Text-to-Speech
To achieve this task, we construct a synthetic dataset and develop an effective framework.
Pushing the limits of self-supervised speaker verification using regularized distillation framework
A range of experiments conducted on the VoxCeleb datasets demonstrate the superiority of the regularized DINO framework in speaker verification.
Deep Representation Decomposition for Rate-Invariant Speaker Verification
While promising performance for speaker verification has been achieved by deep speaker embeddings, the advantage would reduce in the case of speaking-style variability.
Reformulating Speaker Diarization as Community Detection With Emphasis On Topological Structure
In this paper we propose to view clustering-based diarization as a community detection problem.
Graph Convolutional Network Based Semi-Supervised Learning on Multi-Speaker Meeting Data
In this work, we present a GCN-based approach for semi-supervised learning.
Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
Through this formulation, we propose the speaker embedding-aware neural diarization (SEND) framework, where a speech encoder, a speaker encoder, two similarity scorers, and a post-processing network are jointly optimized to predict the encoded labels according to the similarities between speech features and speaker embeddings.
Ranked #1 on
Speaker Diarization
on AliMeeting
Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information
In this paper, we reformulate this task as a single-label prediction problem by encoding the multi-speaker labels with power set.
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences
We propose a novel Pooling Network (PoNet) for token mixing in long sequences with linear complexity.
BeamTransformer: Microphone Array-based Overlapping Speech Detection
We propose BeamTransformer, an efficient architecture to leverage beamformer's edge in spatial filtering and transformer's capability in context sequence modeling.
Measuring daily-life fear perception change: a computational study in the context of COVID-19
COVID-19, as a global health crisis, has triggered the fear emotion with unprecedented intensity.
A Real-time Speaker Diarization System Based on Spatial Spectrum
In this paper we describe a speaker diarization system that enables localization and identification of all speakers present in a conversation or meeting.
Estimating air quality co-benefits of energy transition using machine learning
Applications of the framework with Chinese data reveal highly heterogeneous health benefits of reducing fossil fuel use in different sectors and regions in China with a mean of \$34/tCO2 and a standard deviation of \$84/tCO2.
