Search Results for author:
Found 56 papers, 10 papers with code
Exploring the limits of decoder-only models trained on public speech recognition corpora
The emergence of industrial-scale speech recognition (ASR) models such as Whisper and USM, trained on 1M hours of weakly labelled and 12M hours of audio only proprietary data respectively, has led to a stronger need for large scale public ASR corpora and competitive open source pipelines.
Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization
In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term {bi-level joint unsupervised and supervised training (BL-JUST)}.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Soft Random Sampling: A Theoretical and Empirical Analysis
Soft random sampling (SRS) is a simple yet effective approach for efficient training of large-scale deep neural networks when dealing with massive data.
Semi-Autoregressive Streaming ASR With Label Context
Non-autoregressive (NAR) modeling has gained significant interest in speech processing since these models achieve dramatically lower inference time than autoregressive (AR) models while also achieving good transcription accuracy.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages
Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each.
High-Dimensional Smoothed Entropy Estimation via Dimensionality Reduction
We study the problem of overcoming exponential sample complexity in differential entropy estimation under Gaussian convolutions.
C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval
Inspired by the fact that English text-video retrieval outperforms other languages, we train a student model using input text in different languages to match the cross-modal predictions from teacher models using input text in English.
VQ-T: RNN Transducers using Vector-Quantized Prediction Network States
Beam search, which is the dominant ASR decoding algorithm for end-to-end models, generates tree-structured hypotheses.
Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantization
We report on aggressive quantization strategies that greatly accelerate inference of Recurrent Neural Network Transducers (RNN-T).
Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems
Recent advances in End-to-End (E2E) Spoken Language Understanding (SLU) have been primarily due to effective pretraining of speech representations.
Towards End-to-End Integration of Dialog History for Improved Spoken Language Understanding
In the absence of gold transcripts to fine-tune an ASR model, our model outperforms this baseline by a significant margin of 10% absolute F1 score.
Improving Generalization of Deep Neural Network Acoustic Models with Length Perturbation and N-best Based Label Smoothing
We introduce two techniques, length perturbation and n-best based label smoothing, to improve generalization of deep neural network (DNN) acoustic models for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems
In this paper, we propose a novel text representation and training methodology that allows E2E SLU systems to be effectively constructed using these text resources.
Integrating Text Inputs For Training and Adapting RNN Transducer ASR Models
We observe 20-45% relative word error rate (WER) reduction in these settings with this novel LM style customization technique using only unpaired text data from the new domains.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A new data augmentation method for intent classification enhancement and its application on spoken conversation datasets
The NNSI reduces the need for manual labeling by automatically selecting highly-ambiguous samples and labeling them with high accuracy.
Improving End-to-End Models for Set Prediction in Spoken Language Understanding
The goal of spoken language understanding (SLU) systems is to determine the meaning of the input speech signal, unlike speech recognition which aims to produce verbatim transcripts.
Everything at Once - Multi-Modal Fusion Transformer for Video Retrieval
In this work, we present a multi-modal, modality agnostic fusion transformer that learns to exchange information between multiple modalities, such as video, audio, and text, and integrate them into a fused representation in a joined multi-modal embedding space.
Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval
Multi-modal learning from video data has seen increased attention recently as it allows to train semantically meaningful embeddings without human annotation enabling tasks like zero-shot retrieval and classification.
Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent
We conduct extensive studies over 18 state-of-the-art DL models/tasks and demonstrate that DPSGD often converges in cases where SSGD diverges for large learning rates in the large batch setting.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Cascaded Multilingual Audio-Visual Learning from Videos
In this paper, we explore self-supervised audio-visual models that learn from instructional videos.
Asynchronous Decentralized Distributed Training of Acoustic Models
Specifically, we study three variants of asynchronous decentralized parallel SGD (ADPSGD), namely, fixed and randomized communication patterns on a ring as well as a delay-by-one scheme.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
4-bit Quantization of LSTM-based Speech Recognition Models
We investigate the impact of aggressive low-precision representations of weights and activations in two families of large LSTM-based architectures for Automatic Speech Recognition (ASR): hybrid Deep Bidirectional LSTM - Hidden Markov Models (DBLSTM-HMMs) and Recurrent Neural Network - Transducers (RNN-Ts).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Reducing Exposure Bias in Training Recurrent Neural Network Transducers
By reducing the exposure bias, we show that we can further improve the accuracy of a high-performance RNNT ASR model and obtain state-of-the-art results on the 300-hour Switchboard dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Integrating Dialog History into End-to-End Spoken Language Understanding Systems
End-to-end spoken language understanding (SLU) systems that process human-human or human-computer interactions are often context independent and process each turn of a conversation independently.
Representation based meta-learning for few-shot spoken intent recognition
Spoken intent detection has become a popular approach to interface with various smart devices with ease.
On the limit of English conversational speech recognition
Compensation of the decoder model with the probability ratio approach allows more efficient integration of an external language model, and we report 5. 9% and 11. 5% WER on the SWB and CHM parts of Hub5'00 with very simple LSTM models.
Ranked #1 on
Speech Recognition
on Switchboard + Hub500
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos
Multimodal self-supervised learning is getting more and more attention as it allows not only to train large networks without human supervision but also to search and retrieve data across various modalities.
RNN Transducer Models For Spoken Language Understanding
We present a comprehensive study on building and adapting RNN transducer (RNN-T) models for spoken language understanding(SLU).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Advancing RNN Transducer Technology for Speech Recognition
The techniques pertain to architectural changes, speaker adaptation, language model fusion, model combination and general training recipe.
Federated Acoustic Modeling For Automatic Speech Recognition
In this paper, we investigate federated acoustic modeling using data from multiple clients.
Federated Learning
Speech Recognition
Sound
Distributed, Parallel, and Cluster Computing
Audio and Speech Processing
Why Does Decentralized Training Outperform Synchronous Training In The Large Batch Setting?
We conduct extensive studies over 12 state-of-the-art DL models/tasks and demonstrate that DPSGD consistently outperforms SSGD in the large batch setting; and DPSGD converges in cases where SSGD diverges for large learning rates.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
End-to-end spoken language understanding using transformer networks and self-supervised pre-trained features
Transformer networks and self-supervised pre-training have consistently delivered state-of-art results in the field of natural language processing (NLP); however, their merits in the field of spoken language understanding (SLU) still need further investigation.
Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systems
Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the text-to-intent data is converted into speech-to-intent data using a multi-speaker text-to-speech system.
End-to-End Spoken Language Understanding Without Full Transcripts
For our speech-to-entities experiments on the ATIS corpus, both the CTC and attention models showed impressive ability to skip non-entity words: there was little degradation when trained on just entities versus full transcripts.
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos
Further, we propose a tri-modal model that jointly processes raw audio, video, and text captions from videos to learn a multi-modal semantic embedding space useful for text-video retrieval.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Improving Efficiency in Large-Scale Decentralized Distributed Training
Decentralized Parallel SGD (D-PSGD) and its asynchronous variant Asynchronous Parallel SGD (AD-PSGD) is a family of distributed learning algorithms that have been demonstrated to perform well for large-scale deep learning tasks.
Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard
It is generally believed that direct sequence-to-sequence (seq2seq) speech recognition models are competitive with hybrid models only when a large amount of data, at least a thousand hours, is available for training.
Ranked #2 on
Speech Recognition
on swb_hub_500 WER fullSWBCH
A Kolmogorov Complexity Approach to Generalization in Deep Learning
Using this condition, we formulate an optimization problem to learn a more general classification function.
Challenging the Boundaries of Speech Recognition: The MALACH Corpus
This paper proposes that the community place focus on the MALACH corpus to develop speech recognition systems that are more robust with respect to accents, disfluencies and emotional speech.
A Highly Efficient Distributed Deep Learning System For Automatic Speech Recognition
On commonly used public SWB-300 and SWB-2000 ASR datasets, ADPSGD can converge with a batch size 3X as large as the one used in SSGD, thus enable training at a much larger scale.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Estimating Information Flow in DNNs
We then develop a rigorous estimator for I(X;T) in noisy DNNs and observe compression in various models.
English Broadcast News Speech Recognition by Humans and Machines
With recent advances in deep learning, considerable attention has been given to achieving automatic speech recognition performance close to human performance on tasks like conversational telephone speech (CTS) recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Distributed Deep Learning Strategies For Automatic Speech Recognition
We show that we can train the LSTM model using ADPSGD in 14 hours with 16 NVIDIA P100 GPUs to reach a 7. 6% WER on the Hub5- 2000 Switchboard (SWB) test set and a 13. 1% WER on the CallHome (CH) test set.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Understanding Unequal Gender Classification Accuracy from Face Images
Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender.
Estimating Information Flow in Deep Neural Networks
We then develop a rigorous estimator for $I(X;T)$ in noisy DNNs and observe compression in various models.
Beyond Backprop: Online Alternating Minimization with Auxiliary Variables
Despite significant recent advances in deep neural networks, training them remains a challenge due to the highly non-convex nature of the objective function.
Building competitive direct acoustics-to-word models for English conversational speech recognition
This is because A2W models recognize words from speech without any decoder, pronunciation lexicon, or externally-trained language model, making training and decoding with such models simple.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Kernel Approximation Methods for Speech Recognition
First, in order to reduce the number of random features required by kernel models, we propose a simple but effective method for feature selection.
End-to-End ASR-free Keyword Search from Speech
The first sub-system is a recurrent neural network (RNN)-based acoustic auto-encoder trained to reconstruct the audio through a finite-dimensional representation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Comparison between Deep Neural Nets and Kernel Acoustic Models for Speech Recognition
We study large-scale kernel methods for acoustic modeling and compare to DNNs on performance metrics related to both acoustic modeling and recognition.
Very Deep Multilingual Convolutional Neural Networks for LVCSR
However, CNNs in LVCSR have not kept pace with recent advances in other domains where deeper neural networks provide superior performance.
Ranked #17 on
Speech Recognition
on Switchboard + Hub500
How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets
The computational complexity of kernel methods has often been a major barrier for applying them to large-scale learning problems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improvements to deep convolutional neural networks for LVCSR
We find that with these improvements, particularly with fMLLR and dropout, we are able to achieve an additional 2-3% relative improvement in WER on a 50-hour Broadcast News task over our previous best CNN baseline.
Accelerating Hessian-free optimization for deep neural networks by implicit preconditioning and sampling
This study aims at speeding up Hessian-free training, both by means of decreasing the amount of data used for training, as well as through reduction of the number of Krylov subspace solver iterations used for implicit estimation of the Hessian.
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Most current speech recognition systems use hidden Markov models (HMMs) to deal with the temporal variability of speech and Gaussian mixture models to determine how well each state of each HMM fits a frame or a short window of frames of coefficients that represents the acoustic input.
