Search Results for author:
Found 59 papers, 23 papers with code
ASVspoof 5: Design, Collection and Validation of Resources for Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech
The database contains attacks generated with 32 different algorithms, also crowdsourced, and optimised to varying degrees using new surrogate detection models.
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech
Neural codecs have become crucial to recent speech and audio generation research.
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild
This paper introduces SpoofCeleb, a dataset designed for Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV), utilizing source data from real-world conditions and spoofing attacks generated by Text-To-Speech (TTS) systems also trained on the same real-world data.
Speaker-IPL: Unsupervised Learning of Speaker Characteristics with i-Vector based Pseudo-Labels
To this end, we show that the simple, well-studied, and established i-vector generative model is enough to bootstrap the IPL process for the unsupervised learning of speaker representations.
Text-To-Speech Synthesis In The Wild
The recent literature nonetheless shows efforts to train TTS systems using data collected in the wild.
The VoxCeleb Speaker Recognition Challenge: A Retrospective
In this paper, we provide a review of these challenges that covers: what they explored; the methods developed by the challenge participants and how these evolved; and also the current state of the field for speaker verification and diarisation.
ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale
ASVspoof 5 is the fifth edition in a series of challenges that promote the study of speech spoofing and deepfake attacks, and the design of detection solutions.
Beyond Silence: Bias Analysis through Loss and Asymmetric Approach in Audio Anti-Spoofing
Our investigations highlight the significant differences in training dynamics between the two classes, emphasizing the need for future research to focus on robust modeling of the bonafide class.
On the Evaluation of Speech Foundation Models for Spoken Language Understanding
To answer this, we perform an extensive evaluation of multiple supervised and self-supervised SFMs using several evaluation protocols: (i) frozen SFMs with a lightweight prediction head, (ii) frozen SFMs with a complex prediction head, and (iii) fine-tuned SFMs with a lightweight prediction head.
To what extent can ASV systems naturally defend against spoofing attacks?
The current automatic speaker verification (ASV) task involves making binary decisions on two types of trials: target and non-target.
Beyond Performance Plateaus: A Comprehensive Study on Scalability in Speech Enhancement
Deep learning-based speech enhancement (SE) models have achieved impressive performance in the past decade.
a-DCF: an architecture agnostic metric with application to spoofing-robust speaker verification
Spoofing detection is today a mainstream research topic.
TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
We propose a novel Tri-Modal Translation (TMT) model that translates between arbitrary modalities spanning speech, image, and text.
OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer
Recent studies have highlighted the importance of fully open foundation models.
ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models
First, we provide an open-source platform for researchers in the speaker recognition community to effortlessly build models.
Ranked #1 on
Speaker Recognition
on VoxCeleb1
(using extra training data)
Improving Design of Input Condition Invariant Speech Enhancement
In this paper we propose novel architectures to improve the input condition invariant SE model so that performance in simulated conditions remains competitive while real condition degradation is much mitigated.
AugSumm: towards generalizable speech summarization using synthetic labels from large language model
We tackle this challenge by proposing AugSumm, a method to leverage large language models (LLMs) as a proxy for human annotators to generate augmented summaries for training and evaluation.
Understanding Probe Behaviors through Variational Bounds of Mutual Information
With the success of self-supervised representations, researchers seek a better understanding of the information encapsulated within a representation.
UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions
Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model's behavior and surpassing performance of task-specific models.
Ranked #1 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition
This paper presents a novel framework for joint speaker diarization (SD) and automatic speech recognition (ASR), named SLIDAR (sliding-window diarization-augmented recognition).
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data
Pre-training speech models on large volumes of data has achieved remarkable success.
Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks
We propose a decoder-only language model, VoxtLM, that can perform four tasks: speech recognition, speech synthesis, text generation, and speech continuation.
Encoder-decoder multimodal speaker change detection
The task of speaker change detection (SCD), which detects points where speakers change in an input, is essential for several applications.
Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing
Audio anti-spoofing for automatic speaker verification aims to safeguard users' identities from spoofing attacks.
Towards single integrated spoofing-aware speaker verification embeddings
Second, competitive performance should be demonstrated compared to the fusion of automatic speaker verification (ASV) and countermeasure (CM) embeddings, which outperformed single embedding solutions by a large margin in the SASV2022 challenge.
VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challenge
This paper summarises the findings from the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22), which was held in conjunction with INTERSPEECH 2022.
Absolute decision corrupts absolutely: conservative online speaker diarisation
Our focus lies in developing an online speaker diarisation framework which demonstrates robust performance across diverse domains.
High-resolution embedding extractor for speaker diarisation
Extracted dense frame-level embeddings can each represent a speaker.
Disentangled representation learning for multilingual speaker recognition
The goal of this paper is to learn robust speaker representation for bilingual speaking scenario.
In search of strong embedding extractors for speaker diarisation
First, the evaluation is not straightforward because the features required for better performance differ between speaker verification and diarisation.
Large-scale learning of generalised representations for speaker recognition
We also show that training with proposed large data configurations gives better performance.
Frequency and Multi-Scale Selective Kernel Attention for Speaker Verification
The SKA mechanism allows each convolutional layer to adaptively select the kernel size in a data-driven fashion.
Curriculum learning for self-supervised speaker verification
The goal of this paper is to train effective self-supervised speaker representations without identity labels.
SASV 2022: The First Spoofing-Aware Speaker Verification Challenge
Pre-trained spoofing detection and speaker verification models are provided as open source and are used in two baseline SASV solutions.
Pushing the limits of raw waveform speaker recognition
Our best model achieves an equal error rate of 0. 89%, which is competitive with the state-of-the-art models based on handcrafted features, and outperforms the best model based on raw waveform inputs by a large margin.
Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation
The performance of spoofing countermeasure systems depends fundamentally upon the use of sufficiently representative training data.
Ranked #2 on
Audio Deepfake Detection
on ASVspoof 2021
Multi-scale speaker embedding-based graph attention networks for speaker diarisation
The objective of this work is effective speaker diarisation using multi-scale speaker embeddings.
Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity
The objective of this work is to train noise-robust speaker embeddings adapted for speaker diarisation.
AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks
Artefacts that differentiate spoofed from bona-fide utterances can reside in spectral or temporal domains.
Ranked #1 on
Voice Anti-spoofing
on ASVspoof 2019 - LA
End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection
Artefacts that serve to distinguish bona fide speech from spoofed or deepfake speech are known to reside in specific subbands and temporal segments.
Ranked #6 on
Audio Deepfake Detection
on ASVspoof 2021
Attentive max feature map and joint training for acoustic scene classification
Furthermore, adopting the proposed attentive max feature map, our team placed fourth in the recent DCASE 2021 challenge.
Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System
By learning the reliable intermediate representation of the mean teacher network, we expect that the proposed method can explore more discriminatory embedding spaces and improve the generalization performance of the speaker verification system.
Graph Attention Networks for Anti-Spoofing
This paper reports our use of graph attention networks (GATs) to model these relationships and to improve spoofing detection performance.
Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Network
In this work, we propose an overlapped speech detection system trained as a three-class classifier.
Adapting Speaker Embeddings for Speaker Diarisation
The goal of this paper is to adapt speaker embeddings for solving the problem of speaker diarisation.
Graph Attention Networks for Speaker Verification
The proposed framework inputs segment-wise speaker embeddings from an enrollment and a test utterance and directly outputs a similarity score.
DCASENET: A joint pre-trained deep neural network for detecting and classifying acoustic scenes and events
Single task deep neural networks that perform a target task among diverse cross-related tasks in the acoustic scene and event literature are being developed.
Capturing scattered discriminative information using a deep architecture in acoustic scene classification
Various experiments are conducted using the detection and classification of acoustic scenes and events 2020 task1-a dataset to validate the proposed methods.
Integrated Replay Spoofing-aware Text-independent Speaker Verification
In this paper, we propose two approaches for building an integrated system of speaker verification and presentation attack detection: an end-to-end monolithic approach and a back-end modular approach.
Segment Aggregation for short utterances speaker verification using raw waveforms
The proposed method segments an input utterance into several short utterances and then aggregates the segment embeddings extracted from the segmented inputs to compose a speaker embedding.
Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms
Recent advances in deep learning have facilitated the design of speaker verification systems that directly input raw waveforms.
A study on the role of subsidiary information in replay attack spoofing detection
For addition, we utilize the multi-task learning framework to include subsidiary information to the code.
Self-supervised pre-training with acoustic configurations for replay spoofing detection
Constructing a dataset for replay spoofing detection requires a physical process of playing an utterance and re-recording it, presenting a challenge to the collection of large-scale datasets.
Cosine similarity-based adversarial process
In particular, the adversarial process degrades the performance of the subsidiary model by eliminating the subsidiary information in the input which, in assumption, may degrade the performance of the primary model.
Replay attack detection with complementary high-resolution information using end-to-end DNN for the ASVspoof 2019 Challenge
To detect unrevealed characteristics that reside in a replayed speech, we directly input spectrograms into an end-to-end DNN without knowledge-based intervention.
RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification
In this study, we explore end-to-end deep neural networks that input raw waveforms to improve various aspects: front-end speaker embedding extraction including model architecture, pre-training scheme, additional objective functions, and back-end classification.
End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification
Each speaker basis is designed to represent the corresponding speaker in the process of training deep neural networks.
Short utterance compensation in speaker verification via cosine-based teacher-student learning of speaker embeddings
The short duration of an input utterance is one of the most critical threats that degrade the performance of speaker verification systems.
Replay spoofing detection system for automatic speaker verification using multi-task learning of noise classes
We explore the effectiveness of training a deep neural network simultaneously for replay attack spoofing detection and replay noise classification.
