Search Results for author:
Found 41 papers, 19 papers with code
Linguistic Knowledge Transfer Learning for Speech Enhancement
In this study, we propose the Cross-Modality Knowledge Transfer (CMKT) learning framework, which leverages pre-trained large language models (LLMs) to infuse linguistic knowledge into SE models without requiring text input or LLMs during inference.
Detecting the Undetectable: Assessing the Efficacy of Current Spoof Detection Methods Against Seamless Speech Edits
Neural speech editing advancements have raised concerns about their misuse in spoofing attacks.
NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts
On zero-shot evaluation, NeKo outperforms GPT-3. 5 and Claude-Opus with $15. 5$% to $27. 6$% relative WER reduction in the Hyporadise benchmark.
RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier
In this paper, we present RankUp, a simple yet effective approach that adapts existing semi-supervised classification techniques to enhance the performance of regression tasks.
DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
Recent end-to-end speech language models (SLMs) have expanded upon the capabilities of large language models (LLMs) by incorporating pre-trained speech models.
Generative Speech Foundation Model Pretraining for High-Quality Speech Extraction and Restoration
The proposed method is evaluated on multiple speech restoration tasks, including speech denoising, bandwidth extension, codec artifact removal, and target speaker extraction.
DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
Recent speech language models (SLMs) typically incorporate pre-trained speech models to extend the capabilities from large language models (LLMs).
An Investigation of Incorporating Mamba for Speech Enhancement
This work aims to study a scalable state-space model (SSM), Mamba, for the speech enhancement (SE) task.
Ranked #4 on
Speech Enhancement
on VoiceBank + DEMAND
Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
To improve the robustness of the encoder for SE, a novel self-distillation mechanism combined with adversarial training is introduced.
Multi-objective Non-intrusive Hearing-aid Speech Assessment Model
Furthermore, we demonstrated that incorporating SSL models resulted in greater transferability to OOD dataset.
A Study on Incorporating Whisper for Robust Speech Assessment
This research introduces an enhanced version of the multi-objective speech assessment model--MOSA-Net+, by leveraging the acoustic features from Whisper, a large-scaled weakly supervised model.
Study on the Correlation between Objective Evaluations and Subjective Speech Quality and Intelligibility
Moreover, new objective measures are proposed that combine current objective measures using deep learning techniques to predict subjective quality and intelligibility.
QuAVF: Quality-aware Audio-Visual Fusion for Ego4D Talking to Me Challenge
This technical report describes our QuAVF@NTU-NVIDIA submission to the Ego4D Talking to Me (TTM) Challenge 2023.
Improving Meeting Inclusiveness using Speech Interruption Analysis
In order to drive adoption of its usage to improve inclusiveness (and participation), we present a machine learning-based system that predicts when a meeting participant attempts to obtain the floor, but fails to interrupt (termed a `failed interruption').
Real-time Speech Interruption Analysis: From Cloud to Client Deployment
Meetings are an essential form of communication for all types of organizations, and remote collaboration systems have been much more widely used since the COVID-19 pandemic.
Boosting Self-Supervised Embeddings for Speech Enhancement
We further study the relationship between the noise robustness of SSL representation via clean-noisy distance (CN distance) and the layer importance for SE.
Ranked #20 on
Speech Enhancement
on VoiceBank + DEMAND
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention.
Perceptual Contrast Stretching on Target Feature for Speech Enhancement
Specifically, the contrast of target features is stretched based on perceptual importance, thereby improving the overall SE performance.
Ranked #14 on
Speech Enhancement
on VoiceBank + DEMAND
OSSEM: one-shot speaker adaptive speech enhancement using meta learning
Although deep learning (DL) has achieved notable progress in speech enhancement (SE), further research is still required for a DL-based SE system to adapt effectively and efficiently to particular speakers.
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points
In this paper, a novel sign-exponent-only floating-point network (SEOFP-NET) technique is proposed to compress the model size and accelerate the inference time for speech enhancement, a regression task of speech signal processing.
Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features
Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0. 026 (0. 990 vs 0. 964 in seen noise environments) and 0. 012 (0. 969 vs 0. 957 in unseen noise environments) in perceptual evaluation of speech quality (PESQ) prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC by 0. 021 (0. 985 vs 0. 964 in seen noise environments) and 0. 047 (0. 836 vs 0. 789 in unseen noise environments) in short-time objective intelligibility (STOI) prediction, compared to STOI-Net (based on CRNN), an existing single-task model for STOI prediction.
MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech
Most of the deep learning-based speech enhancement models are learned in a supervised manner, which implies that pairs of noisy and clean speech are required during training.
SpeechBrain: A General-Purpose Speech Toolkit
SpeechBrain is an open-source and all-in-one speech toolkit.
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory.
Ranked #23 on
Speech Enhancement
on VoiceBank + DEMAND
STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model
To overcome this limitation, we propose a deep learning-based non-intrusive speech intelligibility assessment model, namely STOI-Net.
Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement
Speech enhancement (SE) aims to improve speech quality and intelligibility, which are both related to a smooth transition in speech segments that may carry linguistic information, e. g. phones and syllables.
Ranked #23 on
Speech Enhancement
on VoiceBank + DEMAND
CITISEN: A Deep Learning-Based Speech Signal-Processing Mobile Application
The CITISEN provides three functions: speech enhancement (SE), model adaptation (MA), and background noise conversion (BNC), allowing CITISEN to be used as a platform for utilizing and evaluating SE models and flexibly extend the models to address various noise environments and users.
Boosting Objective Scores of a Speech Enhancement Model by MetricGAN Post-processing
The Transformer architecture has demonstrated a superior ability compared to recurrent neural networks in many different natural language processing applications.
iMetricGAN: Intelligibility Enhancement for Speech-in-Noise using Generative Adversarial Network-based Metric Learning
The intelligibility of natural speech is seriously degraded when exposed to adverse noisy environments.
Audio and Speech Processing Sound
Time-Domain Multi-modal Bone/air Conducted Speech Enhancement
Previous studies have proven that integrating video signals, as a complementary modality, can facilitate improved performance for speech enhancement (SE).
Improving the Intelligibility of Electric and Acoustic Stimulation Speech Using Fully Convolutional Networks Based Speech Enhancement
Recently, a time-domain speech enhancement algorithm based on the fully convolutional neural networks (FCN) with a short-time objective intelligibility (STOI)-based objective function (termed FCN(S) in short) has received increasing attention due to its simple structure and effectiveness of restoring clean speech signals from noisy counterparts.
Denoising
Speech Enhancement
+1
Sound
Audio and Speech Processing
Seeing Voices in Noise: A Study of Audiovisual-Enhanced Vocoded Speech Intelligibility in Cochlear Implant Simulation
Speech perception is a key to verbal communication.
Speech Enhancement
Sound
Audio and Speech Processing
Increasing Compactness Of Deep Learning Based Speech Enhancement Models With Parameter Pruning And Quantization Techniques
In addition, a parameter quantization (PQ) technique was applied to reduce the size of a neural network by representing weights with fewer cluster centroids.
MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
Adversarial loss in a conditional generative adversarial network (GAN) is not designed to directly optimize evaluation metrics of a target task, and thus, may not always guide the generator in a GAN to generate data with improved metric scores.
Ranked #37 on
Speech Enhancement
on VoiceBank + DEMAND
Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality
Utilizing a human-perception-related objective function to train a speech enhancement model has become a popular topic recently.
MOSNet: Deep Learning based Objective Assessment for Voice Conversion
In this paper, we propose deep learning-based assessment models to predict human ratings of converted speech.
A study on speech enhancement using exponent-only floating point quantized neural network (EOFP-QNN)
We evaluated the proposed EOFP quantization technique on two types of neural networks, namely, bidirectional long short-term memory (BLSTM) and fully convolutional neural network (FCN), on a speech enhancement task.
Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM
The evaluation of utterance-level quality in Quality-Net is based on the frame-level assessment.
End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks
For example, in measuring speech intelligibility, most of the evaluation metric is based on a short-time objective intelligibility (STOI) measure, while the frame based minimum mean square error (MMSE) between estimated and clean speech is widely used in optimizing the model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Complex spectrogram enhancement by convolutional neural network with multi-metrics learning
This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously.
Raw Waveform-based Speech Enhancement by Fully Convolutional Networks
Because the fully connected layers, which are involved in deep neural networks (DNN) and convolutional neural networks (CNN), may not accurately characterize the local information of speech signals, particularly with high frequency components, we employed fully convolutional layers to model the waveform.
