Search Results for author:
Found 133 papers, 41 papers with code
Chinese Movie Dialogue Question Answering Dataset
This paper also provides a baseline system and shows its performance on this dataset.
A Flexible and Extensible Framework for Multiple Answer Modes Question Answering
Only a few of them adopt several answer generation modules for providing different mechanisms; however, they either lack an aggregation mechanism to merge the answers from various modules, or are too complicated to be implemented with neural networks.
Preoperative Prognosis Assessment of Lumbar Spinal Surgery for Low Back Pain and Sciatica Patients based on Multimodalities and Multimodal Learning
Low back pain (LBP) and sciatica may require surgical therapy when they are symptomatic of severe pain.
Self-supervised based general laboratory progress pretrained model for cardiovascular event detection
In this study, we leveraged self-supervised learning (SSL) and transfer learning to overcome the above-mentioned barriers, transferring patient progress trends in cardiovascular laboratory parameters from prevalent cases to rare or specific cardiovascular events detection.
PreFallKD: Pre-Impact Fall Detection via CNN-ViT Knowledge Distillation
The experiment results show that PreFallKD could boost the student model during the testing phase and achieves reliable F1-score (92. 66%) and lead time (551. 3 ms).
Interpretations of Domain Adaptations via Layer Variational Analysis
The method shows an increase in efficiency and accuracy for domain adaptation.
BASPRO: a balanced script producer for speech corpus collection based on the genetic algorithm
We converted the script into a speech corpus using two text-to-speech systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
On the robustness of non-intrusive speech quality model by adversarial examples
It has been shown recently that deep learning based models are effective on speech quality prediction and could outperform traditional metrics in various perspectives.
Multimodal Forgery Detection Using Ensemble Learning
The recent rapid revolution in Artificial Intelligence (AI) technology has enabled the creation of hyper-realistic deepfakes, and detecting deepfake videos (also known as AIsynthesized videos) has become a critical task.
Ranked #1 on
Multimodal Forgery Detection
on FakeAVCeleb
(using extra training data)
Lip Sync Matters: A Novel Multimodal Forgery Detector
Deepfake technology has advanced a lot, but it is a double-sided sword for the community.
Ranked #1 on
DeepFake Detection
on FakeAVCeleb
Inference and Denoise: Causal Inference-based Neural Speech Enhancement
This study addresses the speech enhancement (SE) task within the causal inference paradigm by modeling the noise presence as an intervention.
T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5
In Spoken language understanding (SLU), a natural solution is concatenating pre-trained speech models (e. g. HuBERT) and pretrained language models (PLM, e. g. T5).
Audio-Visual Speech Enhancement and Separation by Leveraging Multi-Modal Self-Supervised Embeddings
In summary, our results confirm the effectiveness of our proposed model for the AVSS task with proper fine-tuning strategies, demonstrating that multi-modal self-supervised embeddings obtained from AV-HUBERT can be generalized to audio-visual regression tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
CasNet: Investigating Channel Robustness for Speech Separation
In this study, inheriting the use of our previously constructed TAT-2mix corpus, we address the channel mismatch problem by proposing a channel-aware audio separation network (CasNet), a deep learning framework for end-to-end time-domain speech separation.
A Teacher-student Framework for Unsupervised Speech Enhancement Using Noise Remixing Training and Two-stage Inference
The lack of clean speech is a practical challenge to the development of speech enhancement systems, which means that the training of neural network models must be done in an unsupervised manner, and there is an inevitable mismatch between their training criterion and evaluation metric.
ECG Artifact Removal from Single-Channel Surface EMG Using Fully Convolutional Networks
Electrocardiogram (ECG) artifact contamination often occurs in surface electromyography (sEMG) applications when the measured muscles are in proximity to the heart.
Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores.
ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding
To showcase such integration, we performed experiments on carefully designed synthetic datasets for noisy-reverberant multi-channel ST and SLU tasks, which can be used as benchmark corpora for future research.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling
NASTAR uses a feedback mechanism to simulate adaptive training data via a noise extractor and a retrieval model.
EPG2S: Speech Generation and Speech Enhancement based on Electropalatography and Audio Signals using Multimodal Learning
Further, the addition of noisy speech signals is observed to improve quality and intelligibility.
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding
Transformer-based models are widely used in natural language understanding (NLU) tasks, and multimodal transformers have been effective in visual-language tasks.
A Study of Using Cepstrogram for Countermeasure Against Replay Attacks
This study investigated the cepstrogram properties and demonstrated their effectiveness as powerful countermeasures against replay attacks.
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention.
Boosting Self-Supervised Embeddings for Speech Enhancement
We further study the relationship between the noise robustness of SSL representation via clean-noisy distance (CN distance) and the layer importance for SE.
Ranked #7 on
Speech Enhancement
on VoiceBank + DEMAND
MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids
In this study, we propose a multi-branched speech intelligibility prediction model (MBI-Net), for predicting the subjective intelligibility scores of HA users.
Filter-based Discriminative Autoencoders for Children Speech Recognition
Children speech recognition is indispensable but challenging due to the diversity of children's speech.
Perceptual Contrast Stretching on Target Feature for Speech Enhancement
Specifically, the contrast of target features is stretched based on perceptual importance, thereby improving the overall SE performance.
Ranked #4 on
Speech Enhancement
on VoiceBank + DEMAND
Partial Coupling of Optimal Transport for Spoken Language Identification
In order to reduce domain discrepancy to improve the performance of cross-domain spoken language identification (SLID) system, as an unsupervised domain adaptation (UDA) method, we have proposed a joint distribution alignment (JDA) model based on optimal transport (OT).
Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks
However, domain mismatch between training/test situations due to factors, such as speaker, content, channel, and environment, remains a severe problem for speech separation.
Subspace-based Representation and Learning for Phonotactic Spoken Language Recognition
Phonotactic constraints can be employed to distinguish languages by representing a speech utterance as a multinomial distribution or phone events.
Speech-enhanced and Noise-aware Networks for Robust Speech Recognition
In this paper, a noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Continuous Speech for Improved Learning Pathological Voice Disorders
The experiments were conducted on a large-scale database, wherein 1, 045 continuous speech were collected by the speech clinic of a hospital from 2012 to 2019.
When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing
Our experiments on intent classification show that our proposed BERT-QTC model attains competitive experimental results in the Snips and ATIS spoken language datasets.
Partially Fake Audio Detection by Self-attention-based Fake Span Discovery
Also ADD 2022 is the first challenge to propose the partially fake audio detection task.
EMGSE: Acoustic/EMG Fusion for Multimodal Speech Enhancement
Multimodal learning has been proven to be an effective method to improve speech enhancement (SE) performance, especially in challenging situations such as low signal-to-noise ratios, speech noise, or unseen noise types.
A Novel Speech Intelligibility Enhancement Model based on CanonicalCorrelation and Deep Learning
Current deep learning (DL) based approaches to speech intelligibility enhancement in noisy environments are often trained to minimise the feature distance between noise-free speech and enhanced speech signals.
Conditional Diffusion Probabilistic Model for Speech Enhancement
Speech enhancement is a critical component of many user-oriented audio applications, yet current systems still suffer from distorted and unnatural outputs.
A Speech Intelligibility Enhancement Model based on Canonical Correlation and Deep Learning for Hearing-Assistive Technologies
Current deep learning (DL) based approaches to speech intelligibility enhancement in noisy environments are generally trained to minimise the distance between clean and enhanced speech features.
A Novel Temporal Attentive-Pooling based Convolutional Recurrent Architecture for Acoustic Signal Enhancement
To address this problem, we propose to integrate a novel temporal attentive-pooling (TAP) mechanism into a conventional convolutional recurrent neural network, termed as TAP-CRNN.
Predicting the Travel Distance of Patients to Access Healthcare using Deep Neural Networks
The aim of this study is to propose a deep neural network approach to model the complex decision of patient choice in travel distance to access care, which is an important indicator for policymaking in allocating resources.
Toward Real-World Voice Disorder Classification
Conclusion: By deploying factorized convolutional neural networks and domain adversarial training, domain-invariant features can be derived for voice disorder classification with limited resources.
Instrumented shoulder functional assessment using inertial measurement units for frozen shoulder
Moreover, features of subtasks provided subtle information related to clinical conditions that have not been revealed in features of a complete task, especially the defined subtask 1 and 2 of each task.
Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport
This paper presents a novel discriminator-constrained optimal transport network (DOTN) that performs unsupervised domain adaptation for speech enhancement (SE), which is an essential regression task in speech processing.
HASA-net: A non-intrusive hearing-aid speech assessment network
Without the need of a clean reference, non-intrusive speech assessment methods have caught great attention for objective evaluations.
OSSEM: one-shot speaker adaptive speech enhancement using meta learning
Although deep learning (DL) has achieved notable progress in speech enhancement (SE), further research is still required for a DL-based SE system to adapt effectively and efficiently to particular speakers.
SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points
In this paper, a novel sign-exponent-only floating-point network (SEOFP-NET) technique is proposed to compress the model size and accelerate the inference time for speech enhancement, a regression task of speech signal processing.
InQSS: a speech intelligibility and quality assessment model using a multi-task learning network
Speech intelligibility and quality assessment models are essential tools for researchers to evaluate and improve speech processing models.
Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features
In this study, we propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously.
Speech Enhancement Based on Cyclegan with Noise-informed Training
Herein, we investigate a potential limitation of the clean-to-noisy conversion part and propose a novel noise-informed training (NIT) approach to improve the performance of the original CycleGAN SE system.
Speech Enhancement-assisted Voice Conversion in Noisy Environments
Numerous voice conversion (VC) techniques have been proposed for the conversion of voices among different speakers.
MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech
Most of the deep learning-based speech enhancement models are learned in a supervised manner, which implies that pairs of noisy and clean speech are required during training.
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition
We select several pretrained speech representations and present the experimental results on various open-source and publicly available corpora for E2E-ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Classification
In this study, we propose a novel adversarial reprogramming (AR) approach for low-resource spoken command recognition (SCR), and build an AR-SCR system.
Analyzing the Robustness of Unsupervised Speech Recognition
In this work, we further analyze the training robustness of unsupervised ASR on the domain mismatch scenarios in which the domains of unpaired speech and text are different.
Mutual Information Continuity-constrained Estimator
Accordingly, we propose Mutual Information Continuity-constrained Estimator (MICE).
Time Alignment using Lip Images for Frame-based Electrolaryngeal Voice Conversion
Voice conversion (VC) is an effective approach to electrolaryngeal (EL) speech enhancement, a task that aims to improve the quality of the artificial voice from an electrolarynx device.
A Study on Speech Enhancement Based on Diffusion Probabilistic Model
Based on this property, we propose a diffusion probabilistic model-based speech enhancement (DiffuSE) model that aims to recover clean speech signals from noisy signals.
SVSNet: An End-to-end Speaker Voice Similarity Assessment Model
Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention.
Relational Data Selection for Data Augmentation of Speaker-dependent Multi-band MelGAN Vocoder
Nowadays, neural vocoders can generate very high-fidelity speech when a bunch of training data is available.
Speech Recovery for Real-World Self-powered Intermittent Devices
The incompleteness of speech inputs severely degrades the performance of all the related speech signal processing applications.
A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion
First, a powerful parallel sequence-to-sequence model converts the input dysarthric speech into a normal speech of a reference speaker as an intermediate product, and a nonparallel, frame-wise VC model realized with a variational autoencoder then converts the speaker identity of the reference speech back to that of the patient while assumed to be capable of preserving the enhanced quality.
Multimodal Deep Learning Framework for Image Popularity Prediction on Social Media
In this study, motivated by multimodal learning, which uses information from various modalities, and the current success of convolutional neural networks (CNNs) in various fields, we propose a deep learning model, called visual-social convolutional neural network (VSCNN), which predicts the popularity of a posted image by incorporating various types of visual and social features into a unified network model.
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory.
Ranked #9 on
Speech Enhancement
on VoiceBank + DEMAND
The AS-NU System for the M2VoC Challenge
The first track focuses on using a small number of 100 target utterances for voice cloning, while the second track focuses on using only 5 target utterances for voice cloning.
Siamese Neural Network with Joint Bayesian Model Structure for Speaker Verification
However, in most of the discriminative training for SiamNN, only the distribution of pair-wised sample distances is considered, and the additional discriminative information in joint distribution of samples is ignored.
EMA2S: An End-to-End Multimodal Articulatory-to-Speech System
Synthesized speech from articulatory movements can have real-world use for patients with vocal cord disorders, situations requiring silent speech, or in high-noise environments.
Coupling a generative model with a discriminative learning framework for speaker verification
By initializing the two-branch neural network with the generatively learned model parameters of the JB model, we train the model parameters with the pairwise samples as a binary discrimination task.
Attention-based multi-task learning for speech-enhancement and speaker-identification in multi-speaker dialogue scenario
In this study, we propose an attention-based MTL (ATM) approach that integrates MTL and the attention-weighting mechanism to simultaneously realize a multi-model learning structure that performs speech enhancement (SE) and speaker identification (SI).
Unsupervised neural adaptation model based on optimal transport for spoken language identification
By minimizing the classification loss on the training data set with the adaptation loss on both training and testing data sets, the statistical distribution difference between training and testing domains is reduced.
Domain-adaptive Fall Detection Using Deep Adversarial Training
The proposed DAFD can transfer knowledge from the source domain to the target domain by minimizing the domain discrepancy to avoid mismatch problems.
Speech Enhancement with Zero-Shot Model Selection
Experimental results confirmed that the proposed ZMOS approach can achieve better performance in both seen and unseen noise types compared to the baseline systems and other model selection systems, which indicates the effectiveness of the proposed approach in providing robust SE performance.
Deep Learning Based Signal Enhancement of Low-Resolution Accelerometer for Fall Detection Systems
However, the performance of FD systems is diminished owing to low-resolution (LR) accelerometer signals.
SRECG: ECG Signal Super-resolution Framework for Portable/Wearable Devices in Cardiac Arrhythmias Classification
In our experiments, we downsampled the ECG signals from the CPSC2018 dataset and evaluated their HMC accuracies with and without the SRECG.
Speech Enhancement Guided by Contextual Articulatory Information
Previous studies have confirmed that by augmenting acoustic features with the place/manner of articulatory features, the speech enhancement (SE) process can be guided to consider the articulatory properties of the input speech when performing enhancement to attain performance improvements.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model
To overcome this limitation, we propose a deep learning-based non-intrusive speech intelligibility assessment model, namely STOI-Net.
A Study of Incorporating Articulatory Movement Information in Speech Enhancement
Although deep learning algorithms are widely used for improving speech enhancement (SE) performance, the performance remains limited under highly challenging conditions, such as unseen noise or noise signals having low signal-to-noise ratios (SNRs).
Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement
Speech enhancement (SE) aims to improve speech quality and intelligibility, which are both related to a smooth transition in speech segments that may carry linguistic information, e. g. phones and syllables.
Ranked #9 on
Speech Enhancement
on VoiceBank + DEMAND
The Academia Sinica Systems of Voice Conversion for VCC2020
This paper describes the Academia Sinica systems for the two tasks of Voice Conversion Challenge 2020, namely voice conversion within the same language (Task 1) and cross-lingual voice conversion (Task 2).
Improved Lite Audio-Visual Speech Enhancement
Experimental results confirm that compared to conventional AVSE systems, iLAVSE can effectively overcome the aforementioned three practical issues and can improve enhancement performance.
CITISEN: A Deep Learning-Based Speech Signal-Processing Mobile Application
The CITISEN provides three functions: speech enhancement (SE), model adaptation (MA), and background noise conversion (BNC), allowing CITISEN to be used as a platform for utilizing and evaluating SE models and flexibly extend the models to address various noise environments and users.
Incorporating Broad Phonetic Information for Speech Enhancement
In noisy conditions, knowing speech contents facilitates listeners to more effectively suppress background noise components and to retrieve pure speech signals.
Using Deep Learning and Explainable Artificial Intelligence in Patients' Choices of Hospital Levels
This study also used explainable artificial intelligence methods to interpret the contribution of features for the general public and individuals.
Boosting Objective Scores of a Speech Enhancement Model by MetricGAN Post-processing
The Transformer architecture has demonstrated a superior ability compared to recurrent neural networks in many different natural language processing applications.
MIMO Speech Compression and Enhancement Based on Convolutional Denoising Autoencoder
For speech-related applications in IoT environments, identifying effective methods to handle interference noises and compress the amount of data in transmissions is essential to achieve high-quality services.
SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning
The results verify that the SERIL model can effectively adjust itself to new noise environments while overcoming the catastrophic forgetting issue.
Lite Audio-Visual Speech Enhancement
Previous studies have confirmed the effectiveness of incorporating visual information into speech enhancement (SE) systems.
WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement
In WaveCRN, the speech locality feature is captured by a convolutional neural network (CNN), while the temporal sequential property of the locality feature is modeled by stacked simple recurrent units (SRU).
iMetricGAN: Intelligibility Enhancement for Speech-in-Noise using Generative Adversarial Network-based Metric Learning
The intelligibility of natural speech is seriously degraded when exposed to adverse noisy environments.
Audio and Speech Processing Sound
Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion
In this paper, we extend the CDVAE-VC framework by incorporating the concept of adversarial learning, in order to further increase the degree of disentanglement, thereby improving the quality and similarity of converted speech.
Speech Enhancement based on Denoising Autoencoder with Multi-branched Encoders
The DSDT is built based on a prior knowledge of speech and noisy conditions (the speaker, environment, and signal factors are considered in this paper), where each component of the multi-branched encoder performs a particular mapping from noisy to clean speech along the branch in the DSDT.
Cross-scale Attention Model for Acoustic Event Classification
However, a potential limitation of the network is that the discriminative features from the bottom layers (which can model the short-range dependency) are smoothed out in the final representation.
MITAS: A Compressed Time-Domain Audio Separation Network with Parameter Sharing
Our experimental results first confirmed the robustness of our MiTAS on two types of perturbations in mixed audio.
Time-Domain Multi-modal Bone/air Conducted Speech Enhancement
Previous studies have proven that integrating video signals, as a complementary modality, can facilitate improved performance for speech enhancement (SE).
Distributed Microphone Speech Enhancement based on Deep Learning
Speech-related applications deliver inferior performance in complex noise environments.
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques.
Improving the Intelligibility of Electric and Acoustic Stimulation Speech Using Fully Convolutional Networks Based Speech Enhancement
Recently, a time-domain speech enhancement algorithm based on the fully convolutional neural networks (FCN) with a short-time objective intelligibility (STOI)-based objective function (termed FCN(S) in short) has received increasing attention due to its simple structure and effectiveness of restoring clean speech signals from noisy counterparts.
Denoising
Speech Enhancement
+1
Sound
Audio and Speech Processing
Multichannel Speech Enhancement by Raw Waveform-mapping using Fully Convolutional Networks
We also propose an extended version of SDFCN, called the residual SDFCN (termed rSDFCN).
Seeing Voices in Noise: A Study of Audiovisual-Enhanced Vocoded Speech Intelligibility in Cochlear Implant Simulation
Speech perception is a key to verbal communication.
Speech Enhancement
Sound
Audio and Speech Processing
Increasing Compactness Of Deep Learning Based Speech Enhancement Models With Parameter Pruning And Quantization Techniques
In addition, a parameter quantization (PQ) technique was applied to reduce the size of a neural network by representing weights with fewer cluster centroids.
MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
Adversarial loss in a conditional generative adversarial network (GAN) is not designed to directly optimize evaluation metrics of a target task, and thus, may not always guide the generator in a GAN to generate data with improved metric scores.
Ranked #14 on
Speech Enhancement
on VoiceBank + DEMAND
Learning with Learned Loss Function: Speech Enhancement with Quality-Net to Improve Perceptual Evaluation of Speech Quality
Utilizing a human-perception-related objective function to train a speech enhancement model has become a popular topic recently.
Investigation of F0 conditioning and Fully Convolutional Networks in Variational Autoencoder based Voice Conversion
Such hypothesis implies that during the conversion phase, the latent codes and the converted features in VAE based VC are in fact source F0 dependent.
Incorporating Symbolic Sequential Modeling for Speech Enhancement
In this study, the symbolic sequences for acoustic signals are obtained as discrete representations with a Vector Quantized Variational Autoencoder algorithm.
MOSNet: Deep Learning based Objective Assessment for Voice Conversion
In this paper, we propose deep learning-based assessment models to predict human ratings of converted speech.
Boundary-Preserved Deep Denoising of the Stochastic Resonance Enhanced Multiphoton Images
As the rapid growth of high-speed and deep-tissue imaging in biomedical research, it is urgent to find a robust and effective denoising method to retain morphological features for further texture analysis and segmentation.
Refined WaveNet Vocoder for Variational Autoencoder Based Voice Conversion
Conventional WaveNet vocoders are trained with natural acoustic features but conditioned on the converted features in the conversion stage for VC, and such a mismatch often causes significant quality and similarity degradation.
Robustness against the channel effect in pathological voice detection
In this study, we propose a detection system for pathological voice, which is robust against the channel effect.
Speech Enhancement Based on Reducing the Detail Portion of Speech Spectrograms in Modulation Domain via Discrete Wavelet Transform
The presented DWT-based SE method with various scaling factors for the detail part is evaluated with a subset of Aurora-2 database, and the PESQ metric is used to indicate the quality of processed speech signals.
Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech
This paper focuses on using voice conversion (VC) to improve the speech intelligibility of surgical patients who have had parts of their articulators removed.
Voice Conversion Based on Cross-Domain Features Using Variational Auto Encoders
An effective approach to non-parallel voice conversion (VC) is to utilize deep neural networks (DNNs), specifically variational auto encoders (VAEs), to model the latent structure of speech in an unsupervised manner.
A study on speech enhancement using exponent-only floating point quantized neural network (EOFP-QNN)
We evaluated the proposed EOFP quantization technique on two types of neural networks, namely, bidirectional long short-term memory (BLSTM) and fully convolutional neural network (FCN), on a speech enhancement task.
Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM
The evaluation of utterance-level quality in Quality-Net is based on the frame-level assessment.
Noise Adaptive Speech Enhancement using Domain Adversarial Training
The proposed noise adaptive SE system contains an encoder-decoder-based enhancement model and a domain discriminator model.
Sound Audio and Speech Processing
End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks
For example, in measuring speech intelligibility, most of the evaluation metric is based on a short-time objective intelligibility (STOI) measure, while the frame based minimum mean square error (MMSE) between estimated and clean speech is widely used in optimizing the model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks
Precisely speaking, the proposed AVDCNN model is structured as an audio-visual encoder-decoder network, in which audio and visual data are first processed using individual CNNs, and then fused into a joint network to generate enhanced speech (the primary task) and reconstructed images (the secondary task) at the output layer.
Complex spectrogram enhancement by convolutional neural network with multi-metrics learning
This paper aims to address two issues existing in the current speech enhancement methods: 1) the difficulty of phase estimations; 2) a single objective function cannot consider multiple metrics simultaneously.
Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
Building a voice conversion (VC) system from non-parallel speech corpora is challenging but highly valuable in real application scenarios.
Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks
Precisely speaking, the proposed AVDCNN model is structured as an audio-visual encoder-decoder network, in which audio and visual data are first processed using individual CNNs, and then fused into a joint network to generate enhanced speech (the primary task) and reconstructed images (the secondary task) at the output layer.
Raw Waveform-based Speech Enhancement by Fully Convolutional Networks
Because the fully connected layers, which are involved in deep neural networks (DNN) and convolutional neural networks (CNN), may not accurately characterize the local information of speech signals, particularly with high frequency components, we employed fully convolutional layers to model the waveform.
Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora.
Dictionary Update for NMF-based Voice Conversion Using an Encoder-Decoder Network
In this paper, we propose a dictionary update method for Nonnegative Matrix Factorization (NMF) with high dimensional data in a spectral conversion (SC) task.
