Search Results for author:
Found 21 papers, 4 papers with code
MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization
In this paper, we propose a self-supervised music representation learning model for music understanding.
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor
The emergence of novel generative modeling paradigms, particularly audio language models, has significantly advanced the field of song generation.
Gull: A Generative Multifunctional Audio Codec
We introduce Gull, a generative multifunctional audio codec.
ReZero: Region-customizable Sound Extraction
Being a solution to the R-SE task, the proposed ReZero framework includes (1) definitions of different types of spatial regions, (2) methods for region feature extraction and aggregation, and (3) a multi-channel extension of the band-split RNN (BSRNN) model specified for the R-SE task.
Ultra Dual-Path Compression For Joint Echo Cancellation And Noise Suppression
Echo cancellation and noise reduction are essential for full-duplex communication, yet most existing neural networks have high computational costs and are inflexible in tuning model complexity.
The Sound Demixing Challenge 2023 $\unicode{x2013}$ Cinematic Demixing Track
A significant source of this improvement was making the simulated data better match real cinematic audio, which we further investigate in detail.
Probing Deep Speaker Embeddings for Speaker-related Tasks
Results show that all deep embeddings encoded channel and content information in addition to speaker identity, but the extent could vary and their performance on speaker-related tasks can be tremendously different: ECAPA-TDNN is dominant in discriminative tasks, and d-vector leads the guiding tasks, while regulating task is less sensitive to the choice of speaker representations.
High Fidelity Speech Enhancement with Band-split RNN
Despite the rapid progress in speech enhancement (SE) research, enhancing the quality of desired speech in environments with strong noise and interfering speakers remains challenging.
Ranked #3 on
Speech Enhancement
on Deep Noise Suppression (DNS) Challenge
(SI-SDR-WB metric)
Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters
Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks.
Improving Dual-Microphone Speech Enhancement by Learning Cross-Channel Features with Multi-Head Attention
Hand-crafted spatial features, such as inter-channel intensity difference (IID) and inter-channel phase difference (IPD), play a fundamental role in recent deep learning based dual-microphone speech enhancement (DMSE) systems.
Speaker-Aware Mixture of Mixtures Training for Weakly Supervised Speaker Extraction
Dominant researches adopt supervised training for speaker extraction, while the scarcity of ideally clean corpus and channel mismatch problem are rarely considered.
Target Confusion in End-to-end Speaker Extraction: Analysis and Approaches
However, its performance is often inferior to that of a blind source separation (BSS) counterpart with a similar network architecture, due to the auxiliary speaker encoder may sometimes generate ambiguous speaker embeddings.
Learning Decoupling Features Through Orthogonality Regularization
Bearing this in mind, a two-branch deep network (KWS branch and SV branch) with the same network structure is developed and a novel decoupling feature learning method is proposed to push up the performance of KWS and SV simultaneously where speaker-invariant keyword representations and keyword-invariant speaker representations are expected respectively.
Text Anchor Based Metric Learning for Small-footprint Keyword Spotting
Recently proposed metric learning approaches improved the generalizability of models for the KWS task, and 1D-CNN based KWS models have achieved the state-of-the-arts (SOTA) in terms of model size.
Layer Reduction: Accelerating Conformer-Based Self-Supervised Model via Layer Consistency
Transformer-based self-supervised models are trained as feature extractors and have empowered many downstream speech tasks to achieve state-of-the-art performance.
Audio-visual Multi-channel Recognition of Overlapped Speech
Automatic speech recognition (ASR) of overlapped speech remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multi-modal Multi-channel Target Speech Separation
Target speech separation refers to extracting a target speaker's voice from an overlapped audio of simultaneous talkers.
Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning
Hand-crafted spatial features (e. g., inter-channel phase difference, IPD) play a fundamental role in recent deep learning based multi-channel speech separation (MCSS) methods.
Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation
To address these challenges, we propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture in reverberant environments, assisted with directional information of the speaker(s).
A comprehensive study of speech separation: spectrogram vs waveform separation
We study the speech separation problem for far-field data (more similar to naturalistic audio streams) and develop multi-channel solutions for both frequency and time-domain separators with utilizing spectral, spatial and speaker location information.
End-to-End Multi-Channel Speech Separation
This paper extended the previous approach and proposed a new end-to-end model for multi-channel speech separation.
