Search Results for author:
Found 73 papers, 19 papers with code
xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement
While attention-based architectures, such as Conformers, excel in speech enhancement, they face challenges such as scalability with respect to input sequence length.
Ranked #7 on
Speech Enhancement
on VoiceBank + DEMAND
Vocal Tract Length Warped Features for Spoken Keyword Spotting
During testing, the VTL features with different warping factors of a test utterance are scored against the DNN and combined with equal weight.
Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining
This underscores the effectiveness of SSL pretraining in improving the robustness and performance of TS-VAD models in noisy environments.
BiSSL: Bilevel Optimization for Self-Supervised Pre-Training and Fine-Tuning
In this work, we present BiSSL, a first-of-its-kind training framework that introduces bilevel optimization to enhance the alignment between the pretext pre-training and downstream fine-tuning stages in self-supervised learning.
Detecting and Defending Against Adversarial Attacks on Automatic Speech Recognition via Diffusion Models
In this paper, we systematically investigate the use of DMs for defending against adversarial attacks on sentences and examine the effect of varying forward diffusion steps.
Speaker and Style Disentanglement of Speech Based on Contrastive Predictive Coding Supported Factorized Variational Autoencoder
Speech signals encompass various information across multiple levels including content, speaker, and style.
The Effect of Training Dataset Size on Discriminative and Diffusion-Based Speech Enhancement Systems
Diffusion models are typically trained with massive datasets for image generation tasks, but whether this is also required for speech enhancement is unknown.
Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations
Despite its widespread adoption as the prominent neural architecture, the Transformer has spurred several independent lines of work to address its limitations.
Noise-Robust Keyword Spotting through Self-supervised Pretraining
Modern KWS systems are mainly trained using supervised learning methods and require a large amount of labelled data to achieve a good performance.
Hearing-Loss Compensation Using Deep Neural Networks: A Framework and Results From a Listening Test
First, we introduce a framework for emulating auditory models using DNNs, focusing on an auditory-nerve model in the auditory pathway.
How to train your ears: Auditory-model emulation for large-dynamic-range inputs and mild-to-severe hearing losses
Our results show that this new optimization objective significantly improves the emulation performance of deep neural networks across relevant input sound levels and auditory-model frequency channels, without increasing the computational load during inference.
Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions
Our experiments show that self-supervised pretraining not only improves performance in clean conditions, but also yields models which are more robust to adverse conditions compared to purely supervised learning.
PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNs
In this paper, we derive a PAC-Bayes bound on the generalisation gap, in a supervised time-series setting for a special class of discrete-time non-linear dynamical systems.
Investigating the Design Space of Diffusion Models for Speech Enhancement
To address this, we extend this framework to account for the progressive transformation between the clean and noisy speech signals.
Diffusion-Based Speech Enhancement in Matched and Mismatched Conditions Using a Heun-Based Sampler
We show that the proposed system substantially benefits from using multiple databases for training, and achieves superior performance compared to state-of-the-art discriminative models in both matched and mismatched conditions.
Joint Minimum Processing Beamforming and Near-end Listening Enhancement
Results show that the joint optimization can further improve performance compared to the concatenated approach.
Speech inpainting: Context-based speech synthesis guided by video
Audio and visual modalities are inherently connected in speech signals: lip movements and facial expressions are correlated with speech sounds.
Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learners
In this work, we propose a Multi-Window Masked Autoencoder (MW-MAE) fitted with a novel Multi-Window Multi-Head Attention (MW-MHA) module that facilitates the modelling of local-global interactions in every decoder transformer block through attention heads of several distinct local and global windows.
PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss
In this paper we derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) stochastic dynamical systems with inputs.
PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models
In this paper we derive a PAC-Bayesian-Like error bound for a class of stochastic dynamical systems with inputs, namely, for linear time-invariant stochastic state-space models (stochastic LTI systems for short).
Filterbank Learning for Noise-Robust Small-Footprint Keyword Spotting
In the context of keyword spotting (KWS), the replacement of handcrafted speech features by learnable features has not yielded superior KWS performance.
Improved disentangled speech representations using contrastive learning in factorized hierarchical variational autoencoder
For the prior of speaker identity variable, \acrshort{fhvae} assumes it is a Gaussian distribution with an utterance-scale varying mean and a fixed variance.
Leveraging Domain Features for Detecting Adversarial Attacks Against Deep Speech Recognition in Noise
Extensive experiments show that the inverse filter bank features generally perform better in both clean and noisy environments, the detection is effective using either speech or non-speech part, and the acoustic noise can largely degrade the detection performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Minimum Processing Near-end Listening Enhancement
The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment.
Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining
This paper explores the effectiveness of SSL on small models for KWS and establishes that SSL can enhance the performance of small KWS models when labelled data is scarce.
Adversarial Multi-Task Deep Learning for Noise-Robust Voice Activity Detection with Low Algorithmic Delay
Introducing adversarial multi-task learning to the model is observed to increase performance in terms of Area Under Curve (AUC), particularly in noisy environments, while the performance is not degraded at higher SNR levels.
Floor Map Reconstruction Through Radio Sensing and Learning By a Large Intelligent Surface
Environmental scene reconstruction is of great interest for autonomous robotic applications, since an accurate representation of the environment is necessary to ensure safe interaction with robots.
User Localization using RF Sensing: A Performance comparison between LIS and mmWave Radars
Since electromagnetic signals are omnipresent, Radio Frequency (RF)-sensing has the potential to become a universal sensing mechanism with applications in localization, smart-home, retail, gesture recognition, intrusion detection, etc.
Complex Recurrent Variational Autoencoder with Application to Speech Enhancement
This work proposes a complex recurrent VAE framework, specifically in which complex-valued recurrent neural network and L1 reconstruction loss are used.
Disentangled Speech Representation Learning Based on Factorized Hierarchical Variational Autoencoder with Self-Supervised Objective
As a self-supervised objective, autoregressive predictive coding (APC), on the other hand, has been used in extracting meaningful and transferable speech features for multiple downstream tasks.
On Training Targets and Activation Functions for Deep Representation Learning in Text-Dependent Speaker Verification
Furthermore, we study a range of loss functions when speaker identity is used as the training target.
Deep Spoken Keyword Spotting: An Overview
Spoken keyword spotting (KWS) deals with the identification of keywords in audio streams and has become a fast-growing technology thanks to the paradigm shift introduced by deep learning a few years ago.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Joint Far- and Near-End Speech Intelligibility Enhancement based on the Approximated Speech Intelligibility Index
However, the existing optimal mutual information based method requires a complicated system model that includes natural speech variations, and relies on approximations and assumptions of the underlying signal distributions.
Radio Sensing with Large Intelligent Surface for 6G
This paper leverages the potential of Large Intelligent Surface (LIS) for radio sensing in 6G wireless networks.
Explicit construction of the minimum error variance estimator for stochastic LTI state-space systems
In this short article, we showcase the derivation of the optimal (minimum error variance) estimator, when one part of the stochastic LTI system output is not measured but is able to be predicted from the measured system outputs.
Improvement of Noise-Robust Single-Channel Voice Activity Detection with Spatial Pre-processing
It might therefore be advantageous to improve SVAD methods with pre-processing to obtain superior VAD, which is under-explored.
INTERSPEECH 2021 ConferencingSpeech Challenge: Towards Far-field Multi-Channel Speech Enhancement for Video Conferencing
The ConferencingSpeech 2021 challenge is proposed to stimulate research on far-field multi-channel speech enhancement for video conferencing.
PAC-Bayesian theory for stochastic LTI systems
In this paper we derive a PAC-Bayesian error bound for autonomous stochastic LTI state-space models.
Data Generation Using Pass-phrase-dependent Deep Auto-encoders for Text-Dependent Speaker Verification
In this paper, we propose a novel method that trains pass-phrase specific deep neural network (PP-DNN) based auto-encoders for creating augmented data for text-dependent speaker verification (TD-SV).
Vocal Tract Length Perturbation for Text-Dependent Speaker Verification with Autoregressive Prediction Coding
In this letter, we propose a vocal tract length (VTL) perturbation method for text-dependent speaker verification (TD-SV), in which a set of TD-SV systems are trained, one for each VTL factor, and score-level fusion is applied to make a final decision.
Assessing Wireless Sensing Potential with Large Intelligent Surfaces
By treating an LIS as a radio image of the environment relying on the received signal power, we develop techniques to sense the environment, by leveraging the tools of image processing and machine learning.
CC-Loss: Channel Correlation Loss For Image Classification
The loss function is a key component in deep learning models.
Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization
Due to lack of data, overfitting ubiquitously exists in real-world applications of deep neural networks (DNNs).
Audio-Visual Speech Inpainting with Deep Learning
In this paper, we present a deep-learning-based framework for audio-visual speech inpainting, i. e., the task of restoring the missing parts of an acoustic speech signal from reliable audio context and uncorrupted visual information.
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Speech enhancement and speech separation are two related tasks, whose purpose is to extract either one or more target speech signals, respectively, from a mixture of sounds generated by several sources.
UIAI System for Short-Duration Speaker Verification Challenge 2020
Our primary submission to the challenge is the fusion of seven subsystems which yields a normalized minimum detection cost function (minDCF) of 0. 072 and an equal error rate (EER) of 2. 14% on the evaluation set.
A Primer on Large Intelligent Surface (LIS) for Wireless Sensing in an Industrial Setting
One of the beyond-5G developments that is often highlighted is the integration of wireless communication and radio sensing.
Exploring Filterbank Learning for Keyword Spotting
Despite their great performance over the years, handcrafted speech features are not necessarily optimal for any particular speech application.
On Bottleneck Features for Text-Dependent Speaker Verification Using X-vectors
We further investigate the impact of the different bottleneck (BN) features on the performance of x-vectors, including the recently-introduced time-contrastive-learning (TCL) BN features and phone-discriminant BN features.
OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer
A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data.
Vocoder-Based Speech Synthesis from Silent Videos
Both acoustic and visual information influence human perception of speech.
Relaxed N-Pairs Loss for Context-Aware Recommendations of Television Content
This embedding space is used for exploring relevant content in various viewing settings by applying an N-pairs loss objective as well as a relaxed variant proposed in this paper.
Adversarial Example Detection by Classification for Deep Speech Recognition
Machine Learning systems are vulnerable to adversarial attacks and will highly likely produce incorrect outputs under these attacks.
Deep Joint Embeddings of Context and Content for Recommendation
This paper proposes a deep learning-based method for learning joint context-content embeddings (JCCE) with a view to context-aware recommendations, and demonstrate its application in the television domain.
On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement
Finally, we show that a loss function based on scale-invariant signal-to-distortion ratio (SI-SDR) achieves good general performance across a range of popular speech enhancement evaluation metrics, which suggests that SI-SDR is a good candidate as a general-purpose loss function for speech enhancement systems.
Keyword Spotting for Hearing Assistive Devices Robust to External Speakers
Our results show that this multi-task deep residual network is able to achieve a KWS accuracy relative improvement of around 32% with respect to a system that does not deal with external speakers.
rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method
In the end, a posteriori SNR weighted energy difference is applied to the extended pitch segments of the denoised speech signal for detecting voice activity.
Deep-Learning-Based Audio-Visual Speech Enhancement in Presence of Lombard Effect
Regarding speech intelligibility, we find a general tendency of the benefit in training the systems with Lombard speech.
Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification
There are a number of studies about extraction of bottleneck (BN) features from deep neural networks (DNNs)trained to discriminate speakers, pass-phrases and triphone states for improving the performance of text-dependent speaker verification (TD-SV).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Subjective Annotations for Vision-Based Attention Level Estimation
Attention level estimation systems have a high potential in many use cases, such as human-robot interaction, driver modeling and smart home systems, since being able to measure a person's attention level opens the possibility to natural interaction between humans and computers.
On Training Targets and Objective Functions for Deep-Learning-Based Audio-Visual Speech Enhancement
Audio-visual speech enhancement (AV-SE) is the task of improving speech quality and intelligibility in a noisy environment using audio and visual information from a talker.
Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems
Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect.
The Importance of Context When Recommending TV Content: Dataset and Algorithms
Home entertainment systems feature in a variety of usage scenarios with one or more simultaneous users, for whom the complexity of choosing media to consume has increased rapidly over the last decade.
A Parallel/Distributed Algorithmic Framework for Mining All Quantitative Association Rules
We present QARMA, an efficient novel parallel algorithm for mining all Quantitative Association Rules in large multidimensional datasets where items are required to have at least a single common attribute to be specified in the rules single consequent item.
Monaural Speech Enhancement using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure
Finally, we show that the proposed SE system performs on par with a traditional DNN based Short-Time Spectral Amplitude (STSA) SE system in terms of estimated speech intelligibility.
Sound Audio and Speech Processing
Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification
Improving speech system performance in noisy environments remains a challenging task, and speech enhancement (SE) is one of the effective techniques to solve the problem.
Joint Separation and Denoising of Noisy Multi-talker Speech using Recurrent Neural Networks and Permutation Invariant Training
We show that deep bi-directional LSTM RNNs trained using uPIT in noisy environments can improve the Signal-to-Distortion Ratio (SDR) as well as the Extended Short-Time Objective Intelligibility (ESTOI) measure, on the speaker independent multi-talker speech separation and denoising task, for various noise types and Signal-to-Noise Ratios (SNRs).
Sound
Decorrelation of Neutral Vector Variables: Theory and Applications
In this paper, we propose novel strategies for neutral vector variable decorrelation.
Time-Contrastive Learning Based DNN Bottleneck Features for Text-Dependent Speaker Verification
It is well-known that speech signals exhibit quasi-stationary behavior in and only in a short interval, and the TCL method aims to exploit this temporal structure.
Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks
We evaluated uPIT on the WSJ0 and Danish two- and three-talker mixed-speech separation tasks and found that uPIT outperforms techniques based on Non-negative Matrix Factorization (NMF) and Computational Auditory Scene Analysis (CASA), and compares favorably with Deep Clustering (DPCL) and the Deep Attractor Network (DANet).
DNN Filter Bank Cepstral Coefficients for Spoofing Detection
In order to improve the reliability of speaker verification systems, we develop a new filter bank based cepstral feature, deep neural network filter bank cepstral coefficients (DNN-FBCC), to distinguish between natural and spoofed speech.
Incorporating Pass-Phrase Dependent Background Models for Text-Dependent Speaker Verification
In this paper, we propose pass-phrase dependent background models (PBMs) for text-dependent (TD) speaker verification (SV) to integrate the pass-phrase identification process into the conventional TD-SV system, where a PBM is derived from a text-independent background model through adaptation using the utterances of a particular pass-phrase.
Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation
We propose a novel deep learning model, which supports permutation invariant training (PIT), for speaker independent multi-talker speech separation, commonly known as the cocktail-party problem.
