Search Results for author:
Found 109 papers, 36 papers with code
Mitigating the Diminishing Effect of Elastic Weight Consolidation
Elastic weight consolidation (EWC, Kirkpatrick et al. 2017) is a promising approach to addressing catastrophic forgetting in sequential training.
Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms
The ability of countermeasure models to generalize from seen speech synthesis methods to unseen ones has been investigated in the ASVspoof challenge.
Investigating Range-Equalizing Bias in Mean Opinion Score Ratings of Synthesized Speech
Mean Opinion Score (MOS) is a popular measure for evaluating synthesized speech.
Hiding speaker's sex in speech using zero-evidence speaker representation in an analysis/synthesis pipeline
The use of modern vocoders in an analysis/synthesis pipeline allows us to investigate high-quality voice conversion that can be used for privacy purposes.
Outlier-Aware Training for Improving Group Accuracy Disparities
Methods addressing spurious correlations such as Just Train Twice (JTT, arXiv:2107. 09044v2) involve reweighting a subset of the training set to maximize the worst-group accuracy.
Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders
To make better use of pairs of bona fide and spoofed data, this study introduces a contrastive feature loss that can be plugged into the standard training criterion.
Analysis of Master Vein Attacks on Finger Vein Recognition Systems
The results raise the alarm about the robustness of such systems and suggest that master vein attacks should be considered an important security measure.
Spoofing-Aware Attention based ASV Back-end with Multiple Enrollment Utterances and a Sampling Strategy for the SASV Challenge 2022
Current state-of-the-art automatic speaker verification (ASV) systems are vulnerable to presentation attacks, and several countermeasures (CMs), which distinguish bona fide trials from spoofing ones, have been explored to protect ASV.
Joint Speaker Encoder and Neural Back-end Model for Fully End-to-End Automatic Speaker Verification with Multiple Enrollment Utterances
Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring.
The VoicePrivacy 2020 Challenge Evaluation Plan
The VoicePrivacy Challenge aims to promote the development of privacy preservation tools for speech technology by gathering a new community to define the tasks of interest and the evaluation methodology, and benchmarking solutions through a series of challenges.
The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance
Since the short spoofed speech segments to be embedded by attackers are of variable length, six different temporal resolutions are considered, ranging from as short as 20 ms to as large as 640 ms. Third, we propose a new CM that enables the simultaneous use of the segment-level labels at different temporal resolutions as well as utterance-level labels to execute utterance- and segment-level detection at the same time.
The VoicePrivacy 2022 Challenge Evaluation Plan
Participants apply their developed anonymization systems, run evaluation scripts and submit objective evaluation results and anonymized speech data to the organizers.
Joint Noise Reduction and Listening Enhancement for Full-End Speech Enhancement
Speech enhancement (SE) methods mainly focus on recovering clean speech from noisy input.
Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation
The performance of spoofing countermeasure systems depends fundamentally upon the use of sufficiently representative training data.
Robust Deepfake On Unrestricted Media: Generation And Detection
Recent advances in deep learning have led to substantial improvements in deepfake generation, resulting in fake media with a more realistic appearance.
Optimizing Tandem Speaker Verification and Anti-Spoofing Systems
As automatic speaker verification (ASV) systems are vulnerable to spoofing attacks, they are typically used in conjunction with spoofing countermeasure (CM) systems to improve security.
A Practical Guide to Logical Access Voice Presentation Attack Detection
Presentation attack detection (PAD) for ASV, or speech anti-spoofing, is therefore indispensable.
Effectiveness of Detection-based and Regression-based Approaches for Estimating Mask-Wearing Ratio
However, there is still a lack of comprehensive research on both methodologies and datasets.
Investigating self-supervised front ends for speech spoofing countermeasures
Self-supervised speech model is a rapid progressing research topic, and many pre-trained models have been released and used in various down stream tasks.
LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech
An effective approach to automatically predict the subjective rating for synthetic speech is to train on a listening test dataset with human-annotated scores.
Revisiting Speech Content Privacy
In this paper, we discuss an important aspect of speech privacy: protecting spoken content.
LaughNet: synthesizing laughter utterances from waveform silhouettes and a single laughter example
Emotional and controllable speech synthesis is a topic that has received much attention.
Estimating the confidence of speech spoofing countermeasure
On the ASVspoof2019 logical access database, the results demonstrate that an energy-based estimator and a neural-network-based one achieved acceptable performance in identifying unknown attacks in the test set.
Generalization Ability of MOS Prediction Networks
Automatic methods to predict listener opinions of synthesized speech remain elusive since listeners, systems being evaluated, characteristics of the speech, and even the instructions given and the rating scale all vary from test to test.
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
Are end-to-end text-to-speech (TTS) models over-parametrized?
DDS: A new device-degraded speech dataset for speech enhancement
A large and growing amount of speech content in real-life scenarios is being recorded on consumer-grade devices in uncontrolled environments, resulting in degraded speech quality.
Master Face Attacks on Face Recognition Systems
Previous work has proven the existence of master faces, i. e., faces that match multiple enrolled templates in face recognition systems, and their existence extends the ability of presentation attacks.
ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
In addition to a continued focus upon logical and physical access tasks in which there are a number of advances compared to previous editions, ASVspoof 2021 introduces a new task involving deepfake speech detection.
ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan
The automatic speaker verification spoofing and countermeasures (ASVspoof) challenge series is a community-led initiative which aims to promote the consideration of spoofing and the development of countermeasures.
The VoicePrivacy 2020 Challenge: Results and findings
We provide a systematic overview of the challenge design with an analysis of submitted systems and evaluation results.
OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wild
To promote these new tasks, we have created the first large-scale dataset posing a high level of challenges that is designed with face-wise rich annotations explicitly for face forgery detection and segmentation, namely OpenForensics.
Use of speaker recognition approaches for learning and evaluating embedding representations of musical instrument sounds
Constructing an embedding space for musical instrument sounds that can meaningfully represent new and unseen instruments is important for downstream music generation tasks such as multi-instrument synthesis and timbre transfer.
SVSNet: An End-to-end Speaker Voice Similarity Assessment Model
Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention.
Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance
Generally speaking, the main objective when training a neural speech synthesis system is to synthesize natural and expressive speech from the output layer of the neural network without much attention given to the hidden layers.
Visualizing Classifier Adjacency Relations: A Case Study in Speaker Verification and Voice Anti-Spoofing
Whether it be for results summarization, or the analysis of classifier fusion, some means to compare different classifiers can often provide illuminating insight into their behaviour, (dis)similarity or complementarity.
A Multi-Level Attention Model for Evidence-Based Fact Checking
Evidence-based fact checking aims to verify the truthfulness of a claim against evidence extracted from textual sources.
Exploring Disentanglement with Multilingual and Monolingual VQ-VAE
This work examines the content and usefulness of disentangled phone and speaker representations from two separately trained VQ-VAE systems: one trained on multilingual data and another trained on monolingual data.
Fashion-Guided Adversarial Attack on Person Segmentation
It generates adversarial textures learned from fashion style images and then overlays them on the clothing regions in the original image to make all persons in the image invisible to person segmentation networks.
Multi-Metric Optimization using Generative Adversarial Networks for Near-End Speech Intelligibility Enhancement
The intelligibility of speech severely degrades in the presence of environmental noise and reverberation.
An Initial Investigation for Detecting Partially Spoofed Audio
By definition, partially-spoofed utterances contain a mix of both spoofed and bona fide segments, which will likely degrade the performance of countermeasures trained with entirely spoofed utterances.
Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
Probabilistic linear discriminant analysis (PLDA) or cosine similarity have been widely used in traditional speaker verification systems as back-end techniques to measure pairwise similarities.
Ranked #1 on
Speaker Verification
on CN-CELEB
ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech
The ASVspoof initiative was conceived to spearhead research in anti-spoofing for automatic speaker verification (ASV).
Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis
We explore pretraining strategies including choice of base corpus with the aim of choosing the best strategy for zero-shot multi-speaker end-to-end synthesis.
Learning Disentangled Phone and Speaker Representations in a Semi-Supervised VQ-VAE Paradigm
Additionally, phones can be recognized from sub-phone VQ codebook indices in our semi-supervised VQ-VAE better than self-supervised with global conditions.
An Investigation of the Relation Between Grapheme Embeddings and Pronunciation for Tacotron-based Systems
Thanks to this property, we show that grapheme embeddings learned by Tacotron models can be useful for tasks such as grapheme-to-phoneme conversion and control of the pronunciation in synthetic speech.
End-to-End Text-to-Speech using Latent Duration based on VQ-VAE
Explicit duration modeling is a key to achieving robust and efficient alignment in text-to-speech synthesis (TTS).
Latent linguistic embedding for cross-lingual text-to-speech and voice conversion
As the recently proposed voice cloning system, NAUTILUS, is capable of cloning unseen voices using untranscribed speech, we investigate the feasibility of using it to develop a unified cross-lingual TTS/VC system.
Viable Threat on News Reading: Generating Biased News Using Natural Language Models
Recent advancements in natural language generation has raised serious concerns.
Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentals
Recent years have seen growing efforts to develop spoofing countermeasures (CMs) to protect automatic speaker verification (ASV) systems from being deceived by manipulated or artificial inputs.
Generating Master Faces for Use in Performing Wolf Attacks on Face Recognition Systems
In this work, we demonstrated that wolf (generic) faces, which we call "master faces," can also compromise face recognition systems and that the master face concept can be generalized in some cases.
NAUTILUS: a Versatile Voice Cloning System
By using a multi-speaker speech corpus to train all requisite encoders and decoders in the initial training stage, our system can clone unseen voices using untranscribed speech of target speakers on the basis of the backpropagation algorithm.
Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesis
Our experiments suggest that a) a neural sequence-to-sequence TTS system should have a sufficient number of model parameters to produce high quality speech, b) it should also use a powerful encoder when it takes characters as inputs, and c) the encoder still has a room for improvement and needs to have an improved architecture to learn supra-segmental features more appropriately.
The Privacy ZEBRA: Zero Evidence Biometric Recognition Assessment
Mounting privacy legislation calls for the preservation of privacy in speech technology, though solutions are gravely lacking.
Cryptography and Security Audio and Speech Processing
Design Choices for X-vector Based Speaker Anonymization
The recently proposed x-vector based anonymization scheme converts any input voice into that of a random pseudo-speaker.
Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS?
This is followed by an analysis on synthesis quality, speaker and dialect similarity, and a remark on the effectiveness of our speaker augmentation approach.
Introducing the VoicePrivacy Initiative
The VoicePrivacy initiative aims to promote the development of privacy preservation tools for speech technology by gathering a new community to define the tasks of interest and the evaluation methodology, and benchmarking solutions through a series of challenges.
Noise Tokens: Learning Neural Noise Templates for Environment-Aware Speech Enhancement
In recent years, speech enhancement (SE) has achieved impressive progress with the success of deep neural networks (DNNs).
Audio and Speech Processing
iMetricGAN: Intelligibility Enhancement for Speech-in-Noise using Generative Adversarial Network-based Metric Learning
The intelligibility of natural speech is seriously degraded when exposed to adverse noisy environments.
Audio and Speech Processing Sound
An initial investigation on optimizing tandem speaker verification and countermeasure systems using reinforcement learning
The spoofing countermeasure (CM) systems in automatic speaker verification (ASV) are not typically used in isolation of each other.
Detecting and Correcting Adversarial Images Using Image Processing Operations
Deep neural networks (DNNs) have achieved excellent performance on several tasks and have been widely applied in both academia and industry.
Transformation of low-quality device-recorded speech to high-quality speech using improved SEGAN model
Nowadays vast amounts of speech data are recorded from low-quality recorder devices such as smartphones, tablets, laptops, and medium-quality microphones.
Sound Audio and Speech Processing
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques.
A Method for Identifying Origin of Digital Images Using a Convolution Neural Network
The rapid development of deep learning techniques has created new challenges in identifying the origin of digital images because generative adversarial networks and variational autoencoders can create plausible digital images whose contents are not present in natural scenes.
Security of Facial Forensics Models Against Adversarial Attacks
We experimentally demonstrated the existence of individual adversarial perturbations (IAPs) and universal adversarial perturbations (UAPs) that can lead a well-performed FFM to misbehave.
Effect of choice of probability distribution, randomness, and search methods for alignment modeling in sequence-to-sequence text-to-speech synthesis using hard alignment
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods.
Use of a Capsule Network to Detect Fake Images and Videos
In this paper, we introduce a capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning.
Transferring neural speech waveform synthesizers to musical instrument sounds generation
Recent neural waveform synthesizers such as WaveNet, WaveGlow, and the neural-source-filter (NSF) model have shown good performance in speech synthesis despite their different methods of waveform generation.
Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings
While speaker adaptation for end-to-end speech synthesis using speaker embeddings can produce good speaker similarity for speakers seen during training, there remains a gap for zero-shot adaptation to unseen speakers.
Audio and Speech Processing
Bootstrapping non-parallel voice conversion from speaker-adaptive text-to-speech
Voice conversion (VC) and text-to-speech (TTS) are two tasks that share a similar objective, generating speech with a target voice.
Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments
The advantages of our approach are that we can simplify many modules for the soft attention and that we can train the end-to-end TTS model using a single likelihood function.
Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
Advanced neural language models (NLMs) are widely used in sequence generation tasks because they are able to produce fluent and meaningful sentences.
A Unified Speaker Adaptation Method for Speech Synthesis using Transcribed and Untranscribed Speech with Backpropagation
In this study, we propose a novel speech synthesis model, which can be adapted to unseen speakers by fine-tuning part of or all of the network using either transcribed or untranscribed speech.
Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
The output of one branch of the decoder is used for segmenting the manipulated regions while that of the other branch is used for reconstructing the input, which helps improve overall performance.
Speaker Anonymization Using X-vector and Neural Waveform Models
One solution to mitigate these concerns involves the concealing of speaker identities before the sharing of speech data.
Neural source-filter waveform models for statistical parametric speech synthesis
Other models such as Parallel WaveNet and ClariNet bring together the benefits of AR and IAF-based models and train an IAF model by transferring the knowledge from a pre-trained AR teacher to an IAF student without any sequential transformation.
MOSNet: Deep Learning based Objective Assessment for Voice Conversion
In this paper, we propose deep learning-based assessment models to predict human ratings of converted speech.
GELP: GAN-Excited Linear Prediction for Speech Synthesis from Mel-spectrogram
Recent advances in neural network -based text-to-speech have reached human level naturalness in synthetic speech.
Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora
When the available data of a target speaker is insufficient to train a high quality speaker-dependent neural text-to-speech (TTS) system, we can combine data from multiple speakers and train a multi-speaker TTS model instead.
Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform
Recently, we proposed short-time Fourier transform (STFT)-based loss functions for training a neural speech waveform model.
Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks.
Introduction to Voice Presentation Attack Detection and Recent Advances
Over the past few years significant progress has been made in the field of presentation attack detection (PAD) for automatic speaker recognition (ASV).
Identifying Computer-Translated Paragraphs using Coherence Features
We have developed a method for extracting the coherence features from a paragraph by matching similar words in its sentences.
Attentive Filtering Networks for Audio Replay Attack Detection
In this work, we propose our replay attacks detection system - Attentive Filtering Network, which is composed of an attention-based filtering mechanism that enhances feature representations in both the frequency and time domains, and a ResNet-based classifier.
Waveform generation for text-to-speech synthesis using pitch-synchronous multi-scale generative adversarial networks
The state-of-the-art in text-to-speech synthesis has recently improved considerably due to novel neural waveform generation methods, such as WaveNet.
Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language
Towards end-to-end Japanese speech synthesis, we extend Tacotron to systems with self-attention to capture long-term dependencies related to pitch accents and compare their audio quality with classical pipeline systems under various conditions to show their pros and cons.
Audiovisual speaker conversion: jointly and simultaneously transforming facial expression and acoustic characteristics
Transforming the facial and acoustic features together makes it possible for the converted voice and facial expressions to be highly correlated and for the generated target speaker to appear and sound natural.
STFT spectral loss for training a neural speech waveform model
This paper proposes a new loss using short-time Fourier transform (STFT) spectra for the aim of training a high-performance neural speech waveform model that predicts raw continuous speech waveform samples directly.
Neural source-filter-based waveform model for statistical parametric speech synthesis
Neural waveform models such as the WaveNet are used in many recent text-to-speech systems, but the original WaveNet is quite slow in waveform generation because of its autoregressive (AR) structure.
Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos
Recent advances in media generation techniques have made it easier for attackers to create forged images and videos.
MesoNet: a Compact Facial Video Forgery Detection Network
This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face.
Multimodal speech synthesis architecture for unsupervised speaker adaptation
Two new training schemes for the new architecture are also proposed in this paper.
Investigating accuracy of pitch-accent annotations in neural network-based speech synthesis and denoising effects
We investigated the impact of noisy linguistic features on the performance of a Japanese speech synthesis system based on neural network that uses WaveNet vocoder.
Wasserstein GAN and Waveform Loss-based Acoustic Model Training for Multi-speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder
In order to reduce the mismatched characteristics between natural and generated acoustic features, we propose frameworks that incorporate either a conditional generative adversarial network (GAN) or its variant, Wasserstein GAN with gradient penalty (WGAN-GP), into multi-speaker speech synthesis that uses the WaveNet vocoder.
Scaling and bias codes for modeling speaker-adaptive DNN-based speech synthesis systems
Most neural-network based speaker-adaptive acoustic models for speech synthesis can be categorized into either layer-based or input-code approaches.
Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
Generating versatile and appropriate synthetic speech requires control over the output expression separate from the spoken text.
Speaker-independent raw waveform model for glottal excitation
Recent speech technology research has seen a growing interest in using WaveNets as statistical vocoders, i. e., generating speech waveforms from acoustic features.
t-DCF: a Detection Cost Function for the Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification
The two challenge editions in 2015 and 2017 involved the assessment of spoofing countermeasures (CMs) in isolation from ASV using an equal error rate (EER) metric.
A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment
As a supplement to subjective results for the 2018 Voice Conversion Challenge (VCC'18) data, we configure a standard constant-Q cepstral coefficient CM to quantify the extent of processing artifacts.
Transformation on Computer-Generated Facial Image to Avoid Detection by Spoofing Detector
Making computer-generated (CG) images more difficult to detect is an interesting problem in computer graphics and security.
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
We present the Voice Conversion Challenge 2018, designed as a follow up to the 2016 edition with the aim of providing a common framework for evaluating and comparing different state-of-the-art voice conversion (VC) systems.
A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis
Recent advances in speech synthesis suggest that limitations such as the lossy nature of the amplitude spectrum with minimum phase approximation and the over-smoothing effect in acoustic modeling can be overcome by using advanced machine learning approaches.
Speech waveform synthesis from MFCC sequences with generative adversarial networks
This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis.
High-quality nonparallel voice conversion based on cycle-consistent adversarial network
Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data.
Complex-Valued Restricted Boltzmann Machine for Direct Speech Parameterization from Complex Spectra
In contrast, the proposed feature extractor using the CRBM directly encodes the complex spectra (or another complex-valued representation of the complex spectra) into binary-valued latent features (hidden units).
Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama's voice using GAN, WaveNet and low-quality found data
Thanks to the growing availability of spoofing databases and rapid advances in using them, systems for detecting voice spoofing attacks are becoming more and more capable, and error rates close to zero are being reached for the ASVspoof2015 database.
Continuous Expressive Speaking Styles Synthesis based on CVSM and MR-HMM
This paper introduces a continuous system capable of automatically producing the most adequate speaking style to synthesize a desired target text.
Deep Denoising Auto-encoder for Statistical Speech Synthesis
This paper proposes a deep denoising auto-encoder technique to extract better acoustic features for speech synthesis.
