Search Results for author:
Found 75 papers, 19 papers with code
Detecting Neurocognitive Disorders through Analyses of Topic Evolution and Cross-modal Consistency in Visual-Stimulated Narratives
DTM-based approach validated the effectiveness of dynamic topic consistency as a macrostructural metric (F1=0. 61, AUC=0. 78).
Effective and Efficient Mixed Precision Quantization of Speech Foundation Models
This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage.
Structured Speaker-Deficiency Adaptation of Foundation Models for Dysarthric and Elderly Speech Recognition
Experiments on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest structured speaker-deficiency adaptation of HuBERT and Wav2vec2-conformer models consistently outperforms baseline SFMs using either: a) no adapters; b) global adapters shared among all speakers; or c) single attribute adapters modelling speaker or deficiency labels alone by statistically significant WER reductions up to 3. 01% and 1. 50% absolute (10. 86% and 6. 94% relative) on the two tasks respectively.
Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models
Using GPT-4 in the ICKR system can increase of 3. 5% absolute (3. 8% relative) on the Librig2p dataset.
Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTC
SACTC is a tailored CTC variant for multi-talker scenarios, it explicitly models speaker disentanglement by constraining the encoder to represent different speakers' tokens at specific time frames.
Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions
Recent advancements in large language models (LLMs) have revolutionized various domains, bringing significant progress and new opportunities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR
Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.
Exploring SSL Discrete Tokens for Multilingual ASR
With the advancement of Self-supervised Learning (SSL) in speech-related tasks, there has been growing interest in utilizing discrete tokens generated by SSL for automatic speech recognition (ASR), as they offer faster processing techniques.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System
In this study, we propose a pioneering approach to empower Whisper, which is a speech foundation model, to tackle joint multi-talker and target-talker speech recognition tasks.
Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation
The application of data-intensive automatic speech recognition (ASR) technologies to dysarthric and elderly adult speech is confronted by their mismatch against healthy and nonaged voices, data scarcity and large speaker-level variability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition
Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; and the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets.
GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement
Notably, ASR models trained on GigaSpeech 2 can reduce the word error rate for Thai, Indonesian, and Vietnamese on our challenging and realistic YouTube test set by 25% to 40% compared to the Whisper large-v3 model, with merely 10% model parameters.
Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition
Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria.
Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask
This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems.
One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model
We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model.
WavLLM: Towards Robust and Adaptive Speech Large Language Model
In this work, we introduce WavLLM, a robust and adaptive speech large language model with dual encoders, and a prompt-aware LoRA weight adapter, optimized by a two-stage curriculum learning approach.
Cross-Speaker Encoding Network for Multi-Talker Speech Recognition
To the best of our knowledge, this work represents an early effort to integrate SIMO and SISO for multi-talker speech recognition.
Towards Automatic Data Augmentation for Disordered Speech Recognition
Automatic recognition of disordered speech remains a highly challenging task to date due to data scarcity.
Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition
Accurate recognition of cocktail party speech containing overlapping speakers, noise and reverberation remains a highly challenging task to date.
Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition
Automatic recognition of disordered and elderly speech remains highly challenging tasks to date due to data scarcity.
Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems
Rich sources of variability in natural speech present significant challenges to current data intensive speech recognition technologies.
Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systems
Current ASR systems are mainly trained and evaluated at the utterance level.
Use of Speech Impairment Severity for Dysarthric Speech Recognition
A key challenge in dysarthric speech recognition is the speaker-level diversity attributed to both speaker-identity associated factors such as gender, and speech impairment severity.
A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition
Experimental results based on the ACII Challenge 2022 dataset demonstrate the superior performance of the proposed system and the effectiveness of considering multiple relationships using hierarchical regression chain models.
Ranked #1 on
Vocal Bursts Intensity Prediction
on HUME-VB
Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection
This paper presents several efficient methods to extract better AD-related cues from high-level acoustic and linguistic features.
Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition
Experiments conducted on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest TDNN and Conformer ASR systems integrated domain adapted wav2vec2. 0 models consistently outperform the standalone wav2vec2. 0 models by statistically significant WER reductions of 8. 22% and 3. 43% absolute (26. 71% and 15. 88% relative) on the two tasks respectively.
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems
Practical application of unsupervised model-based speaker adaptation techniques to data intensive end-to-end ASR systems is hindered by the scarcity of speaker-level data and performance sensitivity to transcription errors.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Unsupervised Model-based speaker adaptation of end-to-end lattice-free MMI model for speech recognition
Modeling the speaker variability is a key challenge for automatic speech recognition (ASR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition
After LHUC speaker adaptation, the best system using VAE-GAN based augmentation produced an overall WER of 27. 78% on the UASpeech test set of 16 dysarthric speakers, and the lowest published WER of 57. 31% on the subset of speakers with "Very Low" intelligibility.
Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression.
Disentangled Speech Representation Learning for One-Shot Cross-lingual Voice Conversion Using $β$-VAE
We propose an unsupervised learning method to disentangle speech into content representation and speaker identity representation.
Bayesian Neural Network Language Modeling for Speech Recognition
State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex.
Exploring linguistic feature and model combination for speech recognition based automatic AD detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and delay progression.
Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems
Fundamental modelling differences between hybrid and end-to-end (E2E) automatic speech recognition (ASR) systems create large diversity and complementarity among them.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care to delay further progression.
Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition
Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems designed for normal speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition
Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Audio-visual multi-channel speech separation, dereverberation and recognition
Despite the rapid advance of automatic speech recognition (ASR) technologies, accurate recognition of cocktail party speech characterised by the interference from overlapping speakers, background noise and room reverberation remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Neural Architecture Search for Speech Emotion Recognition
Deep neural networks have brought significant advancements to speech emotion recognition (SER).
On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition
Accurate recognition of dysarthric and elderly speech remain challenging tasks to date.
Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition
Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems for normal speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition
Motivated by the spectro-temporal level differences between dysarthric, elderly and normal speech that systematically manifest in articulatory imprecision, decreased volume and clarity, slower speaking rates and increased dysfluencies, novel spectrotemporal subspace basis deep embedding features derived using SVD speech spectrum decomposition are proposed in this paper to facilitate auxiliary feature based speaker adaptation of state-of-the-art hybrid DNN/TDNN and end-to-end Conformer speech recognition systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation
The primary task of ASA fine-tunes the SE with the speech of the target dysarthric speaker to effectively capture identity-related information, and the secondary task applies adversarial training to avoid the incorporation of abnormal speaking patterns into the reconstructed speech, by regularizing the distribution of reconstructed speech to be close to that of reference speech with high quality.
VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion
Though significant progress has been made for speaker-dependent Video-to-Speech (VTS) synthesis, little attention is devoted to multi-speaker VTS that can map silent video to speech, while allowing flexible control of speaker identity, all in a single system.
Investigation of Deep Neural Network Acoustic Modelling Approaches for Low Resource Accented Mandarin Speech Recognition
Meanwhile, approaches of multi-accent modelling including multi-style training, multi-accent decision tree state tying, DNN tandem and multi-level adaptive network (MLAN) tandem hidden Markov model (HMM) modelling are combined and compared in this paper.
Variational Auto-Encoder Based Variability Encoding for Dysarthric Speech Recognition
Dysarthric speech recognition is a challenging task due to acoustic variability and limited amount of available data.
Recent Progress in the CUHK Dysarthric Speech Recognition System
Despite the rapid progress of automatic speech recognition (ASR) technologies in the past few decades, recognition of disordered speech remains a highly challenging task to date.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+4
Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognition
Automatic recognition of disordered speech remains a highly challenging task to date.
Investigation of Data Augmentation Techniques for Disordered Speech Recognition
This paper investigates a set of data augmentation techniques for disordered speech recognition, including vocal tract length perturbation (VTLP), tempo perturbation and speed perturbation.
Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks
State-of-the-art automatic speech recognition (ASR) system development is data and computation intensive.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition
In order to overcome the difficulty in using gradient descent methods to directly estimate discrete quantized weights, alternating direction methods of multipliers (ADMM) are used to efficiently train quantized LMs.
Mixed Precision of Quantization of Transformer Language Models for Speech Recognition
Experiments conducted on Penn Treebank (PTB) and a Switchboard corpus trained LF-MMI TDNN system suggest the proposed mixed precision Transformer quantization techniques achieved model size compression ratios of up to 16 times over the full precision baseline with no recognition performance degradation.
Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers
Index Terms: Language models, Recurrent neural networks, Quantization, Alternating direction methods of multipliers.
Adversarial Data Augmentation for Disordered Speech Recognition
Automatic recognition of disordered speech remains a highly challenging task to date.
Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attacks
This argument motivates the current work that presents a novel, channel-wise gated Res2Net (CG-Res2Net), which modifies Res2Net to enable a channel-wise gating mechanism in the connection between feature groups.
Unsupervised Domain Adaptation for Dysarthric Speech Detection via Domain Adversarial Training and Mutual Information Minimization
Such systems are particularly susceptible to domain mismatch where the training and testing data come from the source and target domains respectively, but the two domains may differ in terms of speech stimuli, disease etiology, etc.
VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion
One-shot voice conversion (VC), which performs conversion across arbitrary speakers with only a single target-speaker utterance for reference, can be effectively achieved by speech representation disentanglement.
Bayesian Transformer Language Models for Speech Recognition
Performance improvements were also obtained on a cross domain LM adaptation task requiring porting a Transformer LM trained on the Switchboard and Fisher data to a low-resource DementiaBank elderly speech corpus.
Bayesian Learning for Deep Neural Network Adaptation
A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences.
Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition
On a third cross domain adaptation task requiring rapidly porting a 1000 hour LibriSpeech data trained system to a small DementiaBank elderly speech corpus, the proposed Bayesian TDNN LF-MMI systems outperformed the baseline system using direct weight fine-tuning by up to 2. 5\% absolute WER reduction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Audio-visual Multi-channel Integration and Recognition of Overlapped Speech
Automatic speech recognition (ASR) technologies have been significantly advanced in the past few decades.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion
Third, a conversion model takes phoneme embeddings and typical prosody features as inputs to generate the converted speech, conditioned on the target DSE that is learned via speaker encoder or speaker adaptation.
Replay and Synthetic Speech Detection with Res2net Architecture
This multiple scaling mechanism significantly improves the countermeasure's generalizability to unseen spoofing attacks.
Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling
During the training stage, an encoder-decoder-based hybrid connectionist-temporal-classification-attention (CTC-attention) phoneme recognizer is trained, whose encoder has a bottle-neck layer.
Understanding the wiring evolution in differentiable neural architecture search
To this end, we pose questions that future differentiable methods for neural wiring discovery need to confront, hoping to evoke a discussion and rethinking on how much bias has been enforced implicitly in existing NAS methods.
Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks
Deep neural networks (DNNs) based automatic speech recognition (ASR) systems are often designed using expert knowledge and empirical evaluation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification
Orthogonal to prior approaches, this work proposes to defend ASV systems against adversarial attacks with a separate detection network, rather than augmenting adversarial data into ASV training.
Audio-visual Multi-channel Recognition of Overlapped Speech
Automatic speech recognition (ASR) of overlapped speech remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Bayesian x-vector: Bayesian Neural Network based x-vector System for Speaker Verification
Our experiment results indicate that the DNN x-vector system could benefit from BNNs especially when the mismatch problem is severe for evaluations using out-of-domain data.
DSNAS: Direct Neural Architecture Search without Parameter Retraining
We argue that given a computer vision task for which a NAS method is expected, this definition can reduce the vaguely-defined NAS evaluation to i) accuracy of this task and ii) the total computation consumed to finally obtain a model with satisfying accuracy.
Ranked #18 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Deep segmental phonetic posterior-grams based discovery of non-categories in L2 English speech
And then we explore the non-categories by looking for the SPPGs with more than one peak.
Audio-visual Recognition of Overlapped speech for the LRS2 dataset
Experiments on overlapped speech simulated from the LRS2 dataset suggest the proposed AVSR system outperformed the audio only baseline LF-MMI DNN system by up to 29. 98\% absolute in word error rate (WER) reduction, and produced recognition performance comparable to a more complex pipelined system.
Ranked #5 on
Audio-Visual Speech Recognition
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition (ASR)
+4
Adversarial Attacks on GMM i-vector based Speaker Verification Systems
Experiment results show that GMM i-vector systems are seriously vulnerable to adversarial attacks, and the crafted adversarial samples prove to be transferable and pose threats to neuralnetwork speaker embedding based systems (e. g. x-vector systems).
Future Word Contexts in Neural Network Language Models
Instead of using a recurrent unit to capture the complete future word contexts, a feedforward unit is used to model a finite number of succeeding, future, words.
