Search Results for author:
Found 105 papers, 33 papers with code
A Flexible and Extensible Framework for Multiple Answer Modes Question Answering
Only a few of them adopt several answer generation modules for providing different mechanisms; however, they either lack an aggregation mechanism to merge the answers from various modules, or are too complicated to be implemented with neural networks.
Is Character Trigram Overlapping Ratio Still the Best Similarity Measure for Aligning Sentences in a Paraphrased Corpus?
This paper studies if the character trigram is still a suitable similarity measure for the task of aligning sentences in a paragraph paraphrasing corpus.
Chinese Movie Dialogue Question Answering Dataset
This paper also provides a baseline system and shows its performance on this dataset.
Mining Commonsense and Domain Knowledge from Math Word Problems
We construct two math datasets and show the effectiveness of our algorithms that they can retrieve the required knowledge for problem-solving.
NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aids
To address this challenge, we present NeuroAMP, a novel deep neural network designed for end-to-end, personalized amplification in hearing aids.
MSECG: Incorporating Mamba for Robust and Efficient ECG Super-Resolution
Electrocardiogram (ECG) signals play a crucial role in diagnosing cardiovascular diseases.
How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception
Extensive experiments are conducted on videos from a benchmark multimodal deepfake dataset to evaluate the detection performance of ChatGPT and compare it with the detection capabilities of state-of-the-art multimodal forensic models and humans.
Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights
Deep Learning has been successfully applied in diverse fields, and its impact on deepfake detection is no exception.
Robust Audio-Visual Speech Enhancement: Correcting Misassignments in Complex Environments with Advanced Post-Processing
This paper addresses the prevalent issue of incorrect speech output in audio-visual speech enhancement (AVSE) systems, which is often caused by poor video quality and mismatched training and test data.
Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition
To mitigate this issue, we propose a novel channel-aware data simulation method for robust ASR training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
A Study on Zero-shot Non-intrusive Speech Assessment using Large Language Models
We evaluate the assessment metrics predicted by GPT-4o and GPT-Whisper, examining their correlation with human-based quality and intelligibility assessments and the character error rate (CER) of automatic speech recognition.
Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement
In multichannel speech enhancement, effectively capturing spatial and spectral information across different microphones is crucial for noise reduction.
Exploring the Impact of Data Quantity on ASR in Extremely Low-resource Languages
This study investigates the efficacy of data augmentation techniques for low-resource automatic speech recognition (ASR), focusing on two endangered Austronesian languages, Amis and Seediq.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbation
Cross-domain speech enhancement (SE) is often faced with severe challenges due to the scarcity of noise and background information in an unseen target domain, leading to a mismatch between training and test conditions.
SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Models
In this paper, we propose SVSNet+, a model that integrates pre-trained SFM representations to improve performance in assessing speaker voice similarity.
Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes
Can we humans correctly perceive the authenticity of the content of the videos we watch?
SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data
Second, we propose a new hybrid architecture that merges the cascaded and parallel architectures of SpeechCLIP into a multi-task learning framework.
HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids
Experimental results demonstrate HAAQI-Net's effectiveness, achieving a Linear Correlation Coefficient (LCC) of 0. 9368 , a Spearman's Rank Correlation Coefficient (SRCC) of 0. 9486 , and a Mean Squared Error (MSE) of 0. 0064 and inference time significantly reduces from 62. 52 to 2. 54 seconds.
LC4SV: A Denoising Framework Learning to Compensate for Unseen Speaker Verification Models
The performance of speaker verification (SV) models may drop dramatically in noisy environments.
D4AM: A General Denoising Framework for Downstream Acoustic Models
To our knowledge, this is the first work that deploys an effective combination scheme of regression (denoising) and classification (ASR) objectives to derive a general pre-processor applicable to various unseen ASR systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multi-objective Non-intrusive Hearing-aid Speech Assessment Model
Furthermore, we demonstrated that incorporating SSL models resulted in greater transferability to OOD dataset.
AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Video Deepfake Detection
This study proposes a new method based on a multi-modal self-supervised-learning (SSL) feature extractor to exploit inconsistency between audio and visual modalities for multi-modal video forgery detection.
Ranked #1 on
DeepFake Detection
on FakeAVCeleb
(Accuracy (%) metric)
AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection
For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset.
Ranked #2 on
DeepFake Detection
on FakeAVCeleb
(Accuracy (%) metric)
The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains
We present the second edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthesized and processed speech.
A Study on Incorporating Whisper for Robust Speech Assessment
This research introduces an enhanced version of the multi-objective speech assessment model--MOSA-Net+, by leveraging the acoustic features from Whisper, a large-scaled weakly supervised model.
Deep Complex U-Net with Conformer for Audio-Visual Speech Enhancement
Recent studies have increasingly acknowledged the advantages of incorporating visual data into speech enhancement (SE) systems.
Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata
Automated speech intelligibility assessment is pivotal for hearing aid (HA) development.
Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Model
This study proposes a multi-task pseudo-label learning (MPL)-based non-intrusive speech quality assessment model called MTQ-Net.
BASPRO: a balanced script producer for speech corpus collection based on the genetic algorithm
We converted the script into a speech corpus using two text-to-speech systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multimodal Forgery Detection Using Ensemble Learning
The recent rapid revolution in Artificial Intelligence (AI) technology has enabled the creation of hyper-realistic deepfakes, and detecting deepfake videos (also known as AIsynthesized videos) has become a critical task.
Ranked #1 on
Multimodal Forgery Detection
on FakeAVCeleb
(using extra training data)
Lip Sync Matters: A Novel Multimodal Forgery Detector
Deepfake technology has advanced a lot, but it is a double-sided sword for the community.
Ranked #3 on
DeepFake Detection
on FakeAVCeleb
(Accuracy (%) metric)
A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech
The lack of clean speech is a practical challenge to the development of speech enhancement systems, which means that there is an inevitable mismatch between their training criterion and evaluation metric.
CasNet: Investigating Channel Robustness for Speech Separation
In this study, inheriting the use of our previously constructed TAT-2mix corpus, we address the channel mismatch problem by proposing a channel-aware audio separation network (CasNet), a deep learning framework for end-to-end time-domain speech separation.
Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores.
NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling
NASTAR uses a feedback mechanism to simulate adaptive training data via a noise extractor and a retrieval model.
A Study of Using Cepstrogram for Countermeasure Against Replay Attacks
This study investigated the cepstrogram properties and demonstrated their effectiveness as powerful countermeasures against replay attacks.
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention.
MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids
In this study, we propose a multi-branched speech intelligibility prediction model (MBI-Net), for predicting the subjective intelligibility scores of HA users.
Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks
However, domain mismatch between training/test situations due to factors, such as speaker, content, channel, and environment, remains a severe problem for speech separation.
Generation of Speaker Representations Using Heterogeneous Training Batch Assembly
In traditional speaker diarization systems, a well-trained speaker model is a key component to extract representations from consecutive and partially overlapping segments in a long speech session.
Subspace-based Representation and Learning for Phonotactic Spoken Language Recognition
Phonotactic constraints can be employed to distinguish languages by representing a speech utterance as a multinomial distribution or phone events.
Speech-enhanced and Noise-aware Networks for Robust Speech Recognition
In this paper, a noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Chain-based Discriminative Autoencoders for Speech Recognition
For application to robust speech recognition, we further extend c-DcAE to hierarchical and parallel structures, resulting in hc-DcAE and pc-DcAE.
Partially Fake Audio Detection by Self-attention-based Fake Span Discovery
Also ADD 2022 is the first challenge to propose the partially fake audio detection task.
EMGSE: Acoustic/EMG Fusion for Multimodal Speech Enhancement
Multimodal learning has been proven to be an effective method to improve speech enhancement (SE) performance, especially in challenging situations such as low signal-to-noise ratios, speech noise, or unseen noise types.
HASA-net: A non-intrusive hearing-aid speech assessment network
Without the need of a clean reference, non-intrusive speech assessment methods have caught great attention for objective evaluations.
Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features
Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0. 026 (0. 990 vs 0. 964 in seen noise environments) and 0. 012 (0. 969 vs 0. 957 in unseen noise environments) in perceptual evaluation of speech quality (PESQ) prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC by 0. 021 (0. 985 vs 0. 964 in seen noise environments) and 0. 047 (0. 836 vs 0. 789 in unseen noise environments) in short-time objective intelligibility (STOI) prediction, compared to STOI-Net (based on CRNN), an existing single-task model for STOI prediction.
Speech Enhancement-assisted Voice Conversion in Noisy Environments
Numerous voice conversion (VC) techniques have been proposed for the conversion of voices among different speakers.
Time Alignment using Lip Images for Frame-based Electrolaryngeal Voice Conversion
Voice conversion (VC) is an effective approach to electrolaryngeal (EL) speech enhancement, a task that aims to improve the quality of the artificial voice from an electrolarynx device.
SVSNet: An End-to-end Speaker Voice Similarity Assessment Model
Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention.
Dual-Path Filter Network: Speaker-Aware Modeling for Speech Separation
DPFN is composed of two parts: the speaker module and the separation module.
Relational Data Selection for Data Augmentation of Speaker-dependent Multi-band MelGAN Vocoder
Nowadays, neural vocoders can generate very high-fidelity speech when a bunch of training data is available.
A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion
First, a powerful parallel sequence-to-sequence model converts the input dysarthric speech into a normal speech of a reference speaker as an intermediate product, and a nonparallel, frame-wise VC model realized with a variational autoencoder then converts the speaker identity of the reference speech back to that of the patient while assumed to be capable of preserving the enhanced quality.
Sequence to General Tree: Knowledge-Guided Geometry Word Problem Solving
With the recent advancements in deep learning, neural solvers have gained promising results in solving math word problems.
AlloST: Low-resource Speech Translation without Source Transcription
In this study, we survey methods to improve ST performance without using source transcription, and propose a learning framework that utilizes a language-independent universal phone recognizer.
The AS-NU System for the M2VoC Challenge
The first track focuses on using a small number of 100 target utterances for voice cloning, while the second track focuses on using only 5 target utterances for voice cloning.
Speech Recognition by Simply Fine-tuning BERT
We propose a simple method for automatic speech recognition (ASR) by fine-tuning BERT, which is a language model (LM) trained on large-scale unlabeled text data and can generate rich contextual representations.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speech Enhancement with Zero-Shot Model Selection
Experimental results confirmed that the proposed ZMOS approach can achieve better performance in both seen and unseen noise types compared to the baseline systems and other model selection systems, which indicates the effectiveness of the proposed approach in providing robust SE performance.
STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model
To overcome this limitation, we propose a deep learning-based non-intrusive speech intelligibility assessment model, namely STOI-Net.
The Academia Sinica Systems of Voice Conversion for VCC2020
This paper describes the Academia Sinica systems for the two tasks of Voice Conversion Challenge 2020, namely voice conversion within the same language (Task 1) and cross-lingual voice conversion (Task 2).
Improved Lite Audio-Visual Speech Enhancement
Experimental results confirm that compared to conventional AVSE systems, iLAVSE can effectively overcome the aforementioned three practical issues and can improve enhancement performance.
Lite Audio-Visual Speech Enhancement
Previous studies have confirmed the effectiveness of incorporating visual information into speech enhancement (SE) systems.
SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning
The results verify that the SERIL model can effectively adjust itself to new noise environments while overcoming the catastrophic forgetting issue.
WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement
In WaveCRN, the speech locality feature is captured by a convolutional neural network (CNN), while the temporal sequential property of the locality feature is modeled by stacked simple recurrent units (SRU).
Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion
In this paper, we extend the CDVAE-VC framework by incorporating the concept of adversarial learning, in order to further increase the degree of disentanglement, thereby improving the quality and similarity of converted speech.
Speech Enhancement based on Denoising Autoencoder with Multi-branched Encoders
The DSDT is built based on a prior knowledge of speech and noisy conditions (the speaker, environment, and signal factors are considered in this paper), where each component of the multi-branched encoder performs a particular mapping from noisy to clean speech along the branch in the DSDT.
Distributed Microphone Speech Enhancement based on Deep Learning
Speech-related applications deliver inferior performance in complex noise environments.
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques.
Multichannel Speech Enhancement by Raw Waveform-mapping using Fully Convolutional Networks
We also propose an extended version of SDFCN, called the residual SDFCN (termed rSDFCN).
Improving the Intelligibility of Electric and Acoustic Stimulation Speech Using Fully Convolutional Networks Based Speech Enhancement
Recently, a time-domain speech enhancement algorithm based on the fully convolutional neural networks (FCN) with a short-time objective intelligibility (STOI)-based objective function (termed FCN(S) in short) has received increasing attention due to its simple structure and effectiveness of restoring clean speech signals from noisy counterparts.
Denoising
Speech Enhancement
+1
Sound
Audio and Speech Processing
Improving Automatic Jazz Melody Generation by Transfer Learning Techniques
In this paper, we tackle the problem of transfer learning for Jazz automatic generation.
Investigation of F0 conditioning and Fully Convolutional Networks in Variational Autoencoder based Voice Conversion
Such hypothesis implies that during the conversion phase, the latent codes and the converted features in VAE based VC are in fact source F0 dependent.
MOSNet: Deep Learning based Objective Assessment for Voice Conversion
In this paper, we propose deep learning-based assessment models to predict human ratings of converted speech.
Refined WaveNet Vocoder for Variational Autoencoder Based Voice Conversion
Conventional WaveNet vocoders are trained with natural acoustic features but conditioned on the converted features in the conversion stage for VC, and such a mismatch often causes significant quality and similarity degradation.
Voice Conversion Based on Cross-Domain Features Using Variational Auto Encoders
An effective approach to non-parallel voice conversion (VC) is to utilize deep neural networks (DNNs), specifically variational auto encoders (VAEs), to model the latent structure of speech in an unsupervised manner.
Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM
The evaluation of utterance-level quality in Quality-Net is based on the frame-level assessment.
Noise Adaptive Speech Enhancement using Domain Adversarial Training
The proposed noise adaptive SE system contains an encoder-decoder-based enhancement model and a domain discriminator model.
Sound Audio and Speech Processing
Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks
Precisely speaking, the proposed AVDCNN model is structured as an audio-visual encoder-decoder network, in which audio and visual data are first processed using individual CNNs, and then fused into a joint network to generate enhanced speech (the primary task) and reconstructed images (the secondary task) at the output layer.
Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks
Building a voice conversion (VC) system from non-parallel speech corpora is challenging but highly valuable in real application scenarios.
Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks
Precisely speaking, the proposed AVDCNN model is structured as an audio-visual encoder-decoder network, in which audio and visual data are first processed using individual CNNs, and then fused into a joint network to generate enhanced speech (the primary task) and reconstructed images (the secondary task) at the output layer.
Learning to Distill: The Essence Vector Modeling Framework
The D-EV model not only inherits the advantages of the EV model but also can infer a more robust representation for a given spoken paragraph against imperfect speech recognition.
Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora.
Dictionary Update for NMF-based Voice Conversion Using an Encoder-Decoder Network
In this paper, we propose a dictionary update method for Nonnegative Matrix Factorization (NMF) with high dimensional data in a spectral conversion (SC) task.
Novel Word Embedding and Translation-based Language Modeling for Extractive Speech Summarization
Word embedding methods revolve around learning continuous distributed vector representations of words with neural networks, which can capture semantic and/or syntactic cues, and in turn be used to induce similarity measures among words, sentences and documents in context.
Improved Spoken Document Summarization with Coverage Modeling Techniques
In addition to MMR, there is only a dearth of research concentrating on reducing redundancy or increasing diversity for the spoken document summarization task, as far as we are aware.
Leveraging Word Embeddings for Spoken Document Summarization
Owing to the rapidly growing multimedia content available on the Internet, extractive spoken document summarization, with the purpose of automatically selecting a set of representative sentences from a spoken document to concisely express the most important theme of the document, has been an active area of research and experimentation.
