Search Results for author:
Found 67 papers, 9 papers with code
End-to-End Attention based Text-Dependent Speaker Verification
A new type of End-to-End system for text-dependent speaker verification is presented in this paper.
Large-Scale Domain Adaptation via Teacher-Student Learning
High accuracy speech recognition requires a large amount of transcribed data for supervised training.
Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
Unsupervised domain adaptation of speech signal aims at adapting a well-trained source-domain acoustic model to the unlabeled data from target domain.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Acoustic-To-Word Model Without OOV
However, this type of word-based CTC model suffers from the out-of-vocabulary (OOV) issue as it can only model limited number of words in the output layer and maps all the remaining words into an OOV output node.
Advancing Acoustic-to-Word CTC Model
However, the word-based CTC model suffers from the out-of-vocabulary (OOV) issue as it can only model limited number of words in the output layer and maps all the remaining words into an OOV output node.
Advancing Connectionist Temporal Classification With Attention Modeling
In this study, we propose advancing all-neural speech recognition by directly incorporating attention modeling within the Connectionist Temporal Classification (CTC) framework.
Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
In this method, a student acoustic model and a condition classifier are jointly optimized to minimize the Kullback-Leibler divergence between the output distributions of the teacher and student models, and simultaneously, to min-maximize the condition classification loss.
Speaker-Invariant Training via Adversarial Learning
We propose a novel adversarial multi-task learning scheme, aiming at actively curtailing the inter-talker feature variability while maximizing its senone discriminability so as to enhance the performance of a deep neural network (DNN) based ASR system.
Developing Far-Field Speaker System Via Teacher-Student Learning
In this study, we develop the keyword spotting (KWS) and acoustic model (AM) components in a far-field speaker system.
Layer Trajectory LSTM
In this paper, we propose a layer trajectory LSTM (ltLSTM) which builds a layer-LSTM using all the layer outputs from a standard multi-layer time-LSTM.
Cycle-Consistent Speech Enhancement
In this paper, we propose a cycle-consistent speech enhancement (CSE) in which an additional inverse mapping network is introduced to reconstruct the noisy features from the enhanced ones.
Adversarial Feature-Mapping for Speech Enhancement
To achieve better performance on ASR task, senone-aware (SA) AFM is further proposed in which an acoustic model network is jointly trained with the feature-mapping and discriminator networks to optimize the senone classification loss in addition to the AFM losses.
Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
In particular, we introduce Attention CTC, Self-Attention CTC, Hybrid CTC, and Mixed-unit CTC.
Speaker Adaptation for End-to-End CTC Models
We propose two approaches for speaker adaptation in end-to-end (E2E) automatic speech recognition systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Conditional Teacher-Student Learning
To overcome this problem, we propose a conditional T/S learning scheme, in which a "smart" student model selectively chooses to learn from either the teacher model or the ground truth labels conditioned on whether the teacher can correctly predict the ground truth.
Attentive Adversarial Learning for Domain-Invariant Training
Adversarial domain-invariant training (ADIT) proves to be effective in suppressing the effects of domain variability in acoustic modeling and has led to improved performance in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Adversarial Speaker Verification
The use of deep networks to extract embeddings for speaker recognition has proven successfully.
Adversarial Speaker Adaptation
We propose a novel adversarial speaker adaptation (ASA) scheme, in which adversarial learning is applied to regularize the distribution of deep hidden features in a speaker-dependent (SD) deep neural network (DNN) acoustic model to be close to that of a fixed speaker-independent (SI) DNN acoustic model during adaptation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Encrypted Speech Recognition using Deep Polynomial Networks
One good property of the DPN is that it can be trained on unencrypted speech features in the traditional way.
PyKaldi2: Yet another speech toolkit based on Kaldi and PyTorch
While similar toolkits are available built on top of the two, a key feature of PyKaldi2 is sequence training with criteria such as MMI, sMBR and MPE.
Self-Teaching Networks
Furthermore, the auxiliary loss also works as a regularizer, which improves the generalization capacity of the network.
Improving RNN Transducer Modeling for End-to-End Speech Recognition
In this paper, we improve the RNN-T training in two aspects.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speaker Adaptation for Attention-Based End-to-End Speech Recognition
We propose three regularization-based speaker adaptation approaches to adapt the attention-based encoder-decoder (AED) model with very limited adaptation data from target speakers for end-to-end automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Advances in Online Audio-Visual Meeting Transcription
This increases marginally to 1. 6% when 50% of the attendees are unknown to the system.
Character-Aware Attention-Based End-to-End Speech Recognition
However, as one input to the decoder recurrent neural network (RNN), each WSU embedding is learned independently through context and acoustic information in a purely data-driven fashion.
Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
In this work, we extend the T/S learning to large-scale unsupervised domain adaptation of an attention-based end-to-end (E2E) model through two levels of knowledge transfer: teacher's token posteriors as soft labels and one-best predictions as decoder guidance.
BLK-REW: A Unified Block-based DNN Pruning Framework using Reweighted Regularization Method
Accelerating DNN execution on various resource-limited computing platforms has been a long-standing problem.
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations.
RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices.
High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model
While the community keeps promoting end-to-end models over conventional hybrid models, which usually are long short-term memory (LSTM) models trained with a cross entropy criterion followed by a sequence discriminative training criterion, we argue that such conventional hybrid models can still be significantly improved.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Minimum Latency Training Strategies for Streaming Sequence-to-Sequence ASR
This leads to an inevitable latency during inference.
L-Vector: Neural Label Embedding for Domain Adaptation
We propose a novel neural label embedding (NLE) scheme for the domain adaptation of a deep neural network (DNN) acoustic model with unpaired data samples from source and target domains.
Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition
Recently, the recurrent neural network transducer (RNN-T) architecture has become an emerging trend in end-to-end automatic speech recognition research due to its advantages of being capable for online streaming speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Exploring Transformers for Large-Scale Speech Recognition
While recurrent neural networks still largely define state-of-the-art speech recognition systems, the Transformer network has been proven to be a competitive alternative, especially in the offline condition.
Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability
Because of its streaming nature, recurrent neural network transducer (RNN-T) is a very promising end-to-end (E2E) model that may replace the popular hybrid model for automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speaker Separation Using Speaker Inventories and Estimated Speech
We propose speaker separation using speaker inventories and estimated speech (SSUSIES), a framework leveraging speaker profiles and estimated speech for speaker separation.
On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer
Hybrid Autoregressive Transducer (HAT) is a recently proposed end-to-end acoustic model that extends the standard Recurrent Neural Network Transducer (RNN-T) for the purpose of the external language model (LM) fusion.
Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition
The external language models (LM) integration remains a challenging task for end-to-end (E2E) automatic speech recognition (ASR) which has no clear division between acoustic and language models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Streaming end-to-end multi-talker speech recognition
End-to-end multi-talker speech recognition is an emerging research trend in the speech community due to its vast potential in applications such as conversation and meeting transcriptions.
Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition
The efficacy of external language model (LM) integration with existing end-to-end (E2E) automatic speech recognition (ASR) systems can be improved significantly using the internal language model estimation (ILME) method.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to transcribe the audio as well as identify the speakers for downstream applications.
On Addressing Practical Challenges for RNN-Transducer
The first challenge is solved with a splicing data method which concatenates the speech segments extracted from the source domain data.
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
In this work, we perform LM fusion in the minimum WER (MWER) training of an E2E model to obviate the need for LM weights tuning during inference.
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices.
Diarisation using location tracking with agglomerative clustering
Previous works have shown that spatial location information can be complementary to speaker embeddings for a speaker diarisation task.
Joint speaker diarisation and tracking in switching state-space model
Speakers may move around while diarisation is being performed.
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
ILMA enables a fast text-only adaptation of the E2E model without increasing the run-time computational cost.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition
Hybrid and end-to-end (E2E) systems have their individual advantages, with different error patterns in the speech recognition results.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices.
Endpoint Detection for Streaming End-to-End Multi-talker ASR
Our experimental results based on the 2-speaker LibrispeechMix dataset show that the SURT model can achieve promising EP detection without significantly degradation of the recognition accuracy.
Reverse Engineering of Imperceptible Adversarial Image Perturbations
However, carefully crafted, tiny adversarial perturbations are difficult to recover by optimizing a unilateral RED objective.
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence.
SparCL: Sparse Continual Learning on the Edge
SparCL achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity.
Streaming, fast and accurate on-device Inverse Text Normalization for Automatic Speech Recognition
Automatic Speech Recognition (ASR) systems typically yield output in lexical form.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors
In this paper, we uncover them by model checkpoints' gradients, forming the proposed self-ensemble protection (SEP), which is very effective because (1) learning on examples ignored during normal training tends to yield DNNs ignoring normal examples; (2) checkpoints' cross-model gradients are close to orthogonal, meaning that they are as diverse as DNNs with different architectures.
All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i. e., keeping the difference in speed performance under various execution frequencies as small as possible.
Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Training
We propose gated language experts and curriculum training to enhance multilingual transformer transducer models without requiring language identification (LID) input from users during inference.
Can Adversarial Examples Be Parsed to Reveal Victim Model Information?
We call this 'model parsing of adversarial attacks' - a task to uncover 'arcana' in terms of the concealed VM information in attacks.
DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning
Rehearsal-based approaches are a mainstay of continual learning (CL).
Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognition
Most end-to-end (E2E) speech recognition models are composed of encoder and decoder blocks that perform acoustic and language modeling functions.
Pre-training End-to-end ASR Models with Augmented Speech Samples Queried by Text
When mixing 20, 000 hours augmented speech data generated by our method with 12, 500 hours original transcribed speech data for Italian Transformer transducer model pre-training, we achieve 8. 7% relative word error rate reduction.
Hybrid Attention-based Encoder-decoder Model for Efficient Language Model Adaptation
Attention-based encoder-decoder (AED) speech recognition model has been widely successful in recent years.
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values
The rapid advancement of Large Language Models (LLMs) has attracted much attention to value alignment for their responsible development.
E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
One highly promising direction for enabling flexible real-time on-device image editing is utilizing data distillation by leveraging large-scale text-to-image diffusion models, such as Stable Diffusion, to generate paired datasets used for training generative adversarial networks (GANs).
NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription
The challenge focuses on distant speaker diarization and automatic speech recognition (DASR) in far-field meeting scenarios, with single-channel and known-geometry multi-channel tracks, and serves as a launch platform for two new datasets: First, a benchmarking dataset of 315 meetings, averaging 6 minutes each, capturing a broad spectrum of real-world acoustic conditions and conversational dynamics.
