Search Results for author:
Found 31 papers, 11 papers with code
A Large-Scale Evaluation of Speech Foundation Models
In this work, we establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the paradigm for speech.
EMO-SUPERB: An In-depth Look at Speech Emotion Recognition
Speech emotion recognition (SER) is a pivotal technology for human-computer interaction systems.
Codec-SUPERB: An In-Depth Analysis of Sound Codec Models
The sound codec's dual roles in minimizing data transmission latency and serving as tokenizers underscore its critical importance.
Towards audio language modeling - an overview
Neural audio codecs are initially introduced to compress audio data into compact codes to reduce transmission latency.
Scalable Ensemble-based Detection Method against Adversarial Attacks for speaker verification
Automatic speaker verification (ASV) is highly susceptible to adversarial attacks.
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Audio-visual representation learning aims to develop systems with human-like perception by utilizing correlation between auditory and visual information.
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
To achieve comprehensive coverage of diverse speech tasks and harness instruction tuning, we invite the community to collaborate and contribute, facilitating the dynamic growth of the benchmark.
SpeechGen: Unlocking the Generative Power of Speech Language Models with Prompts
In this paper, we present pioneering research that explores the application of prompt tuning to stimulate speech LMs for various generation tasks, within a unified framework called SpeechGen, with around 10M trainable parameters.
Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
Extending on this, we incorporate a diarization branch into the Sidecar, allowing for unified modeling of both ASR and diarization with a negligible overhead of only 768 parameters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The defender's perspective on automatic speaker verification: An overview
Automatic speaker verification (ASV) plays a critical role in security-sensitive environments.
Rethinking complex-valued deep neural networks for monaural speech enhancement
By comparing complex- and real-valued versions of fundamental building blocks in the recently developed gated convolutional recurrent network (GCRN), we show how different mechanisms for basic blocks affect the performance.
Multimodal Transformer Distillation for Audio-Visual Synchronization
This paper proposed an MTDVocaLiST model, which is trained by our proposed multimodal Transformer distillation (MTD) loss.
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
We present the SUPERB challenge at SLT 2022, which aims at learning self-supervised speech representation for better performance, generalization, and efficiency.
Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
Audio-visual active speaker detection (AVASD) is well-developed, and now is an indispensable front-end for several multi-modal applications.
Adversarial Robustness
Audio-Visual Active Speaker Detection
Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
However, previous works show that state-of-the-art ASV models are seriously vulnerable to voice spoofing attacks, and the recently proposed high-performance spoofing countermeasure (CM) models only focus solely on the standalone anti-spoofing tasks, and ignore the subsequent speaker verification process.
Human-AI Shared Control via Policy Dissection
Inspired by the neuroscience approach to investigate the motor cortex in primates, we develop a simple yet effective frequency-based approach called \textit{Policy Dissection} to align the intermediate representation of the learned neural controller with the kinematic attributes of the agent behavior.
Spoofing-Aware Speaker Verification by Multi-Level Fusion
In the second-level fusion, the CM score and ASV scores directly from ASV systems will be concatenated into a prediction block for the final decision.
Partially Fake Audio Detection by Self-attention-based Fake Span Discovery
Also ADD 2022 is the first challenge to propose the partially fake audio detection task.
The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
This paper describes our speaker diarization system submitted to the Multi-channel Multi-party Meeting Transcription (M2MeT) challenge, where Mandarin meeting data were recorded in multi-channel format for diarization and automatic speech recognition (ASR) tasks.
Characterizing the adversarial vulnerability of speech self-supervised learning
As the paradigm of the self-supervised learning upstream model followed by downstream tasks arouses more attention in the speech community, characterizing the adversarial robustness of such paradigm is of high priority.
Revisiting the Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning
QMIX, a popular MARL algorithm based on the monotonicity constraint, has been used as a baseline for the benchmark environments, such as Starcraft Multi-Agent Challenge (SMAC), Predator-Prey (PP).
Adversarial Sample Detection for Speaker Verification by Neural Vocoders
We also show that the neural vocoder adopted in the detection framework is dataset-independent.
Voting for the right answer: Adversarial defense for speaker verification
Automatic speaker verification (ASV) is a well developed technology for biometric identification, and has been ubiquitous implemented in security-critic applications, such as banking and access control.
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms.
Tackling Variabilities in Autonomous Driving
In this work, we aim to extensively explore the above system design challenges and these challenges motivate us to propose a comprehensive framework that synergistically handles the heterogeneous hardware accelerator design principles, system design criteria, and task scheduling mechanism.
Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
Automatic speaker verification (ASV) is one of the core technologies in biometric identification.
Rethinking the Implementation Matters in Cooperative Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement Learning (MARL) has seen revolutionary breakthroughs with its successful application to multi-agent cooperative tasks such as computer games and robot swarms.
QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learning
Existing methods such as Value Decomposition Network (VDN) and QMIX estimate the value of long-term returns as a scalar that does not contain the information of randomness.
Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
To explore this issue, we proposed to employ Mockingjay, a self-supervised learning based model, to protect anti-spoofing models against adversarial attacks in the black-box scenario.
Defense against adversarial attacks on spoofing countermeasures of ASV
Various forefront countermeasure methods for automatic speaker verification (ASV) with considerable performance in anti-spoofing are proposed in the ASVspoof 2019 challenge.
Adversarial Attacks on Spoofing Countermeasures of automatic speaker verification
High-performance spoofing countermeasure systems for automatic speaker verification (ASV) have been proposed in the ASVspoof 2019 challenge.
