Search Results for author:
Found 100 papers, 27 papers with code
On Controlling Fallback Responses for Grounded Dialogue Generation
We propose a novel framework that automatically generates a control token with the generator to bias the succeeding response towards informativeness for answerable contexts and fallback for unanswerable contexts in an end-to-end manner.
Unsupervised Multi-scale Expressive Speaking Style Modeling with Hierarchical Context Information for Audiobook Speech Synthesis
On top of these, a bi-reference attention mechanism is used to align both local-scale reference style embedding sequence and local-scale context style embedding sequence with corresponding phoneme embedding sequence.
Partner Personas Generation for Dialogue Response Generation
Incorporating personas information allows diverse and engaging responses in dialogue response generation.
Grounded Dialogue Generation with Cross-encoding Re-ranker, Grounding Span Prediction, and Passage Dropout
For the response generator, we use grounding span prediction as an auxiliary task to be jointly trained with the main task of response generation.
Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection
This paper presents several efficient methods to extract better AD-related cues from high-level acoustic and linguistic features.
A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition
Experimental results based on the ACII Challenge 2022 dataset demonstrate the superior performance of the proposed system and the effectiveness of considering multiple relationships using hierarchical regression chain models.
Decision Support System for Chronic Diseases Based on Drug-Drug Interactions
Many patients with chronic diseases resort to multiple medications to relieve various symptoms, which raises concerns about the safety of multiple medication use, as severe drug-drug antagonism can lead to serious adverse effects or even death.
Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition
Experiments conducted on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest TDNN and Conformer ASR systems integrated domain adapted wav2vec2. 0 models consistently outperform the standalone wav2vec2. 0 models by statistically significant WER reductions of 8. 22% and 3. 43% absolute (26. 71% and 15. 88% relative) on the two tasks respectively.
A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker One
Although automatic speech recognition (ASR) can perform well in common non-overlapping environments, sustaining performance in multi-talker overlapping speech recognition remains challenging.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving Rare Words Recognition through Homophone Extension and Unified Writing for Low-resource Cantonese Speech Recognition
Homophone characters are common in tonal syllable-based languages, such as Mandarin and Cantonese.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection
However, PLM domain fine-tuning is commonly based on the masked word or sentence prediction costs that are inconsistent with the back-end AD detection task.
Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations
Moreover, we optimize the training strategy by leveraging more audio to learn MSMCRs better for low-resource languages.
Disentangled Speech Representation Learning for One-Shot Cross-lingual Voice Conversion Using $β$-VAE
We propose an unsupervised learning method to disentangle speech into content representation and speaker identity representation.
Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation
Further experiments on the downstream task of Cross-Lingual Natural Language Inference show that the proposed model achieves significant performance improvement for distant language pairs in downstream tasks compared to state-of-the-art adversarial and non-adversarial models.
Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
Audio-visual active speaker detection (AVASD) is well-developed, and now is an indispensable front-end for several multi-modal applications.
Adversarial Robustness
Audio-Visual Active Speaker Detection
A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTS
A vector-quantized, variational autoencoder (VQ-VAE) based feature analyzer is used to encode Mel spectrograms of speech training data by down-sampling progressively in multiple stages into MSMC Representations (MSMCRs) with different time resolutions, and quantizing them with multiple VQ codebooks, respectively.
Bayesian Neural Network Language Modeling for Speech Recognition
State-of-the-art neural network language models (NNLMs) represented by long short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming highly complex.
Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversion
One-shot voice conversion (VC) with only a single target speaker's speech for reference has become a hot research topic.
Towards Cross-speaker Reading Style Transfer on Audiobook Dataset
This paper aims to introduce a chunk-wise multi-scale cross-speaker style model to capture both the global genre and the local prosody in audiobook speeches.
Exploring linguistic feature and model combination for speech recognition based automatic AD detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and delay progression.
Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care to delay further progression.
Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems
Fundamental modelling differences between hybrid and end-to-end (E2E) automatic speech recognition (ASR) systems create large diversity and complementarity among them.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
However, previous works show that state-of-the-art ASV models are seriously vulnerable to voice spoofing attacks, and the recently proposed high-performance spoofing countermeasure (CM) models only focus solely on the standalone anti-spoofing tasks, and ignore the subsequent speaker verification process.
Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition
Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems designed for normal speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Cross-lingual Word Embeddings in Hyperbolic Space
Cross-lingual word embeddings can be applied to several natural language processing applications across multiple languages.
Audio-visual multi-channel speech separation, dereverberation and recognition
Despite the rapid advance of automatic speech recognition (ASR) technologies, accurate recognition of cocktail party speech characterised by the interference from overlapping speakers, background noise and room reverberation remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Neural Architecture Search for Speech Emotion Recognition
Deep neural networks have brought significant advancements to speech emotion recognition (SER).
A Character-level Span-based Model for Mandarin Prosodic Structure Prediction
In this paper, we propose a span-based Mandarin prosodic structure prediction model to obtain an optimal prosodic structure tree, which can be converted to corresponding prosodic label sequence.
An End-to-end Chinese Text Normalization Model based on Rule-guided Flat-Lattice Transformer
Inspired by Flat-LAttice Transformer (FLAT), we propose an end-to-end Chinese text normalization model, which accepts Chinese characters as direct input and integrates expert knowledge contained in rules into the neural network, both contribute to the superior performance of proposed model for the text normalization task.
Spoofing-Aware Speaker Verification by Multi-Level Fusion
In the second-level fusion, the CM score and ASV scores directly from ASV systems will be concatenated into a prediction block for the final decision.
On-the-fly Feature Based Speaker Adaptation for Dysarthric and Elderly Speech Recognition
Automatic recognition of dysarthric and elderly speech highly challenging tasks to date.
Disentangleing Content and Fine-grained Prosody Information via Hybrid ASR Bottleneck Features for Voice Conversion
In this paper, we proposed an any-to-one VC method using hybrid bottleneck features extracted from CTC-BNFs and CE-BNFs to complement each other advantages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards Expressive Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesis
In this paper, we propose a hierarchical framework to model speaking style from context.
FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement
Previously proposed FullSubNet has achieved outstanding performance in Deep Noise Suppression (DNS) Challenge and attracted much attention.
Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition
Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems for normal speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition
Motivated by the spectro-temporal level differences between dysarthric, elderly and normal speech that systematically manifest in articulatory imprecision, decreased volume and clarity, slower speaking rates and increased dysfluencies, novel spectrotemporal subspace basis deep embedding features derived using SVD speech spectrum decomposition are proposed in this paper to facilitate auxiliary feature based speaker adaptation of state-of-the-art hybrid DNN/TDNN and end-to-end Conformer speech recognition systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speaker Identity Preservation in Dysarthric Speech Reconstruction by Adversarial Speaker Adaptation
The primary task of ASA fine-tunes the SE with the speech of the target dysarthric speaker to effectively capture identity-related information, and the secondary task applies adversarial training to avoid the incorporation of abnormal speaking patterns into the reconstructed speech, by regularizing the distribution of reconstructed speech to be close to that of reference speech with high quality.
VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion
Though significant progress has been made for speaker-dependent Video-to-Speech (VTS) synthesis, little attention is devoted to multi-speaker VTS that can map silent video to speech, while allowing flexible control of speaker identity, all in a single system.
Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks
The research of open-domain dialog systems has been greatly prospered by neural models trained on large-scale corpora, however, such corpora often introduce various safety problems (e. g., offensive languages, biases, and toxic behaviors) that significantly hinder the deployment of dialog systems in practice.
TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening
This is used in the development of TalkTive, a CA which can predict both timing and form of backchanneling during cognitive assessments.
Partially Fake Audio Detection by Self-attention-based Fake Span Discovery
Also ADD 2022 is the first challenge to propose the partially fake audio detection task.
User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems
In this paper, we propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction, by jointly learning User Satisfaction Estimation and Dialogue Act Recognition tasks.
The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
This paper describes our speaker diarization system submitted to the Multi-channel Multi-party Meeting Transcription (M2MeT) challenge, where Mandarin meeting data were recorded in multi-channel format for diarization and automatic speech recognition (ASR) tasks.
Convex Polytope Modelling for Unsupervised Derivation of Semantic Structure for Data-efficient Natural Language Understanding
Popular approaches for Natural Language Understanding (NLU) usually rely on a huge amount of annotated data or handcrafted rules, which is laborious and not adaptive to domain extension.
Toward Self-learning End-to-End Task-Oriented Dialog Systems
In this paper, we study the problem of automatically adapting task bots to changing environments by learning from human-bot interactions with minimum or zero human annotations.
Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition
Emotion recognition is a challenging and actively-studied research area that plays a critical role in emotion-aware human-computer interaction systems.
COLD: A Benchmark for Chinese Offensive Language Detection
To this end, we propose a benchmark --COLD for Chinese offensive language analysis, including a Chinese Offensive Language Dataset --COLDATASET and a baseline detector --COLDETECTOR which is trained on the dataset.
Recent Progress in the CUHK Dysarthric Speech Recognition System
Despite the rapid progress of automatic speech recognition (ASR) technologies in the past few decades, recognition of disordered speech remains a highly challenging task to date.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+4
Spectro-Temporal Deep Features for Disordered Speech Assessment and Recognition
Automatic recognition of disordered speech remains a highly challenging task to date.
Investigation of Data Augmentation Techniques for Disordered Speech Recognition
This paper investigates a set of data augmentation techniques for disordered speech recognition, including vocal tract length perturbation (VTLP), tempo perturbation and speed perturbation.
Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks
State-of-the-art automatic speech recognition (ASR) system development is data and computation intensive.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Mixed Precision of Quantization of Transformer Language Models for Speech Recognition
Experiments conducted on Penn Treebank (PTB) and a Switchboard corpus trained LF-MMI TDNN system suggest the proposed mixed precision Transformer quantization techniques achieved model size compression ratios of up to 16 times over the full precision baseline with no recognition performance degradation.
Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition
In order to overcome the difficulty in using gradient descent methods to directly estimate discrete quantized weights, alternating direction methods of multipliers (ADMM) are used to efficiently train quantized LMs.
Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers
Index Terms: Language models, Recurrent neural networks, Quantization, Alternating direction methods of multipliers.
Characterizing the adversarial vulnerability of speech self-supervised learning
As the paradigm of the self-supervised learning upstream model followed by downstream tasks arouses more attention in the speech community, characterizing the adversarial robustness of such paradigm is of high priority.
Countering Online Hate Speech: An NLP Perspective
This paper presents a holistic conceptual framework on hate-speech NLP countering methods along with a thorough survey on the current progress of NLP for countering online hate speech.
Adversarial Data Augmentation for Disordered Speech Recognition
Automatic recognition of disordered speech remains a highly challenging task to date.
Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attacks
This argument motivates the current work that presents a novel, channel-wise gated Res2Net (CG-Res2Net), which modifies Res2Net to enable a channel-wise gating mechanism in the connection between feature groups.
Adversarial Sample Detection for Speaker Verification by Neural Vocoders
We also show that the neural vocoder adopted in the detection framework is dataset-independent.
VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion
One-shot voice conversion (VC), which performs conversion across arbitrary speakers with only a single target-speaker utterance for reference, can be effectively achieved by speech representation disentanglement.
Unsupervised Domain Adaptation for Dysarthric Speech Detection via Domain Adversarial Training and Mutual Information Minimization
Such systems are particularly susceptible to domain mismatch where the training and testing data come from the source and target domains respectively, but the two domains may differ in terms of speech stimuli, disease etiology, etc.
Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling
However, state-of-the-art context modeling methods in conversational TTS only model the textual information in context with a recurrent neural network (RNN).
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms.
DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion
Singing voice conversion (SVC) is one promising technique which can enrich the way of human-computer interaction by endowing a computer the ability to produce high-fidelity and expressive singing voice.
Out-of-Scope Domain and Intent Classification through Hierarchical Joint Modeling
User queries for a real-world dialog system may sometimes fall outside the scope of the system's capabilities, but appropriate system responses will enable smooth processing throughout the human-computer interaction.
Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing
This paper presents an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances in a domain.
Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis
Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS).
Towards Multi-Scale Style Control for Expressive Speech Synthesis
This paper introduces a multi-scale speech style modeling method for end-to-end expressive speech synthesis.
Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
Automatic speaker verification (ASV) is one of the core technologies in biometric identification.
Bayesian Transformer Language Models for Speech Recognition
Performance improvements were also obtained on a cross domain LM adaptation task requiring porting a Transformer LM trained on the Switchboard and Fisher data to a low-resource DementiaBank elderly speech corpus.
Adversarially learning disentangled speech representations for robust multi-factor voice conversion
To increase the robustness of highly controllable style transfer on multiple factors in VC, we propose a disentangled speech representation learning framework based on adversarial learning.
Creation and Evaluation of a Pre-tertiary Artificial Intelligence (AI) Curriculum
Background: AI4Future is a cross-sector project that engages five major partners - CUHK Faculty of Engineering and Faculty of Education, Hong Kong secondary schools, the government and the AI industry.
Unstructured Knowledge Access in Task-oriented Dialog Modeling using Language Inference, Knowledge Retrieval and Knowledge-Integrative Response Generation
Dialog systems enriched with external knowledge can handle user queries that are outside the scope of the supporting databases/APIs.
Unsupervised Cross-Lingual Speech Emotion Recognition Using DomainAdversarial Neural Network
By using deep learning approaches, Speech Emotion Recog-nition (SER) on a single domain has achieved many excellentresults.
Syntactic representation learning for neural network based TTS with syntactic parse tree traversal
Meanwhile, nuclear-norm maximization loss is introduced to enhance the discriminability and diversity of the embeddings of constituent labels.
Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition
On a third cross domain adaptation task requiring rapidly porting a 1000 hour LibriSpeech data trained system to a small DementiaBank elderly speech corpus, the proposed Bayesian TDNN LF-MMI systems outperformed the baseline system using direct weight fine-tuning by up to 2. 5\% absolute WER reduction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Audio-visual Multi-channel Integration and Recognition of Overlapped Speech
Automatic speech recognition (ASR) technologies have been significantly advanced in the past few decades.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulation
This paper presents FastSVC, a light-weight cross-domain singing voice conversion (SVC) system, which can achieve high conversion performance, with inference speed 4x faster than real-time on CPUs.
Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion
Third, a conversion model takes phoneme embeddings and typical prosody features as inputs to generate the converted speech, conditioned on the target DSE that is learned via speaker encoder or speaker adaptation.
Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input
Non-autoregressive (NAR) transformer models have achieved significantly inference speedup but at the cost of inferior accuracy compared to autoregressive (AR) models in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Replay and Synthetic Speech Detection with Res2net Architecture
This multiple scaling mechanism significantly improves the countermeasure's generalizability to unseen spoofing attacks.
Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling
During the training stage, an encoder-decoder-based hybrid connectionist-temporal-classification-attention (CTC-attention) phoneme recognizer is trained, whose encoder has a bottle-neck layer.
Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks
Deep neural networks (DNNs) based automatic speech recognition (ASR) systems are often designed using expert knowledge and empirical evaluation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams
Recent approaches mainly have following limitations: 1) most speaker-independent methods need handcrafted features that are time-consuming to design or unreliable; 2) there is no convincing method to support multilingual or mixlingual speech as input.
Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification
Orthogonal to prior approaches, this work proposes to defend ASV systems against adversarial attacks with a separate detection network, rather than augmenting adversarial data into ASV training.
Audio-visual Multi-channel Recognition of Overlapped Speech
Automatic speech recognition (ASR) of overlapped speech remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Bayesian x-vector: Bayesian Neural Network based x-vector System for Speaker Verification
Our experiment results indicate that the DNN x-vector system could benefit from BNNs especially when the mismatch problem is severe for evaluations using out-of-domain data.
Defense against adversarial attacks on spoofing countermeasures of ASV
Various forefront countermeasure methods for automatic speaker verification (ASV) with considerable performance in anti-spoofing are proposed in the ASVspoof 2019 challenge.
Deep segmental phonetic posterior-grams based discovery of non-categories in L2 English speech
And then we explore the non-categories by looking for the SPPGs with more than one peak.
Audio-visual Recognition of Overlapped speech for the LRS2 dataset
Experiments on overlapped speech simulated from the LRS2 dataset suggest the proposed AVSR system outperformed the audio only baseline LF-MMI DNN system by up to 29. 98\% absolute in word error rate (WER) reduction, and produced recognition performance comparable to a more complex pipelined system.
Ranked #4 on
Audio-Visual Speech Recognition
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition (ASR)
+4
Adversarial Attacks on GMM i-vector based Speaker Verification Systems
Experiment results show that GMM i-vector systems are seriously vulnerable to adversarial attacks, and the crafted adversarial samples prove to be transferable and pose threats to neuralnetwork speaker embedding based systems (e. g. x-vector systems).
Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks
Our best systems achieve a relative improvement of 11. 9% and 8. 3% on the TIMIT and WSJ tasks respectively.
Adversarial Attacks on Spoofing Countermeasures of automatic speaker verification
High-performance spoofing countermeasure systems for automatic speaker verification (ASV) have been proposed in the ASVspoof 2019 challenge.
Semi-Supervised Graph Classification: A Hierarchical Graph Perspective
We study the node classification problem in the hierarchical graph where a `node' is a graph instance, e. g., a user group in the above example.
Ranked #10 on
Graph Classification
on D&D
Study on Feature Subspace of Archetypal Emotions for Speech Emotion Recognition
Hence, traditional methods may fail to distinguish some of the emotions with just one global feature subspace.
Feature Learning with Gaussian Restricted Boltzmann Machine for Robust Speech Recognition
In this paper, we first present a new variant of Gaussian restricted Boltzmann machine (GRBM) called multivariate Gaussian restricted Boltzmann machine (MGRBM), with its definition and learning algorithm.
