Search Results for author:
Found 73 papers, 6 papers with code
Text Injection for Capitalization and Turn-Taking Prediction in Speech Models
Text injection for automatic speech recognition (ASR), wherein unpaired text-only data is used to supplement paired audio-text data, has shown promising improvements for word error rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improving Joint Speech-Text Representations Without Alignment
The last year has seen astonishing progress in text-prompted image generation premised on the idea of a cross-modal representation space in which the text and image domains are represented jointly.
How to Estimate Model Transferability of Pre-Trained Speech Models?
In this work, we introduce a "score-based assessment" framework for estimating the transferability of pre-trained speech models (PSMs) for fine-tuning target tasks.
Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR
We address this limitation by distilling punctuation knowledge from a bidirectional teacher language model (LM) trained on written, punctuated text.
Mixture-of-Expert Conformer for Streaming Multilingual ASR
We evaluate the proposed model on a set of 12 languages, and achieve an average 11. 9% relative improvement in WER over the baseline.
Modular Domain Adaptation for Conformer-Based Streaming ASR
Speech data from different domains has distinct acoustic and linguistic characteristics.
Lego-Features: Exporting modular encoder features for streaming and deliberation ASR
Overall, they present a modular, powerful and cheap alternative to the standard encoder output, as well as the N-best hypotheses.
Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR
Conformer models maintain a large number of internal states, the vast majority of which are associated with self-attention layers.
A Deliberation-based Joint Acoustic and Text Decoder
We propose a new two-pass E2E speech recognition model that improves ASR performance by training on a combination of paired data and unpaired text data.
Sharing Low Rank Conformer Weights for Tiny Always-On Ambient Speech Recognition Models
Such a model allows us to achieve always-on ambient speech recognition on edge devices with low-memory neural processors.
End-to-End Speech Recognition: A Survey
In the last decade of automatic speech recognition (ASR) research, the introduction of deep learning brought considerable reductions in word error rate of more than 50% relative, compared to modeling without deep learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
UML: A Universal Monolingual Output Layer for Multilingual ASR
Consequently, the UML enables to switch in the interpretation of each output node depending on the language of the input speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition
We propose JEIT, a joint end-to-end (E2E) model and internal language model (ILM) training method to inject large-scale unpaired text into ILM during E2E training which improves rare-word speech recognition.
Efficient Domain Adaptation for Speech Foundation Models
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition
In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
E2E Segmentation in a Two-Pass Cascaded Encoder ASR Model
We explore unifying a neural segmenter with two-pass cascaded encoder ASR into a single model.
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion
Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition
We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues often encountered in training large-scare acoustic models in low-resource scenarios.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Scaling Up Deliberation for Multilingual ASR
In this work, we investigate second-pass deliberation for multilingual speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Turn-Taking Prediction for Natural Conversational Speech
This makes doing speech recognition with conversational speech, including one with multiple queries, a challenging task.
Streaming Intended Query Detection using E2E Modeling for Continued Conversation
In voice-enabled applications, a predetermined hotword isusually used to activate a device in order to attend to the query. However, speaking queries followed by a hotword each timeintroduces a cognitive burden in continued conversations.
A Language Agnostic Multilingual Streaming On-Device ASR System
On-device end-to-end (E2E) models have shown improvements over a conventional model on English Voice Search tasks in both quality and latency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving Deliberation by Text-Only and Semi-Supervised Training
Text-only and semi-supervised training based on audio-only data has gained popularity recently due to the wide availability of unlabeled text and speech data.
Self-Supervised Speech Representation Learning: A Review
Although self-supervised speech representation is still a nascent research area, it is closely related to acoustic word embedding and learning with zero lexical resources, both of which have seen active research for many years.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASR
Improving the performance of end-to-end ASR models on long utterances ranging from minutes to hours in length is an ongoing challenge in speech recognition.
Streaming Align-Refine for Non-autoregressive Deliberation
We propose a streaming non-autoregressive (non-AR) decoding algorithm to deliberate the hypothesis alignment of a streaming RNN-T model.
Improving Rare Word Recognition with LM-aware MWER Training
Language models (LMs) significantly improve the recognition accuracy of end-to-end (E2E) models on words rarely seen during training, when used in either the shallow fusion or the rescoring setups.
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition
We down-select a large corpus of web search queries by a factor of 53x and achieve better LM perplexities than without down-selection.
Improving the fusion of acoustic and text representations in RNN-T
The recurrent neural network transducer (RNN-T) has recently become the mainstream end-to-end approach for streaming automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Joint Unsupervised and Supervised Training for Multilingual ASR
Our average WER of all languages outperforms average monolingual baseline by 33. 3%, and the state-of-the-art 2-stage XLSR by 32%.
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Tied & Reduced RNN-T Decoder
Previous works on the Recurrent Neural Network-Transducer (RNN-T) models have shown that, under some conditions, it is possible to simplify its prediction network with little or no loss in recognition accuracy (arXiv:2003. 07705 [eess. AS], [2], arXiv:2012. 06749 [cs. CL]).
Scaling End-to-End Models for Large-Scale Multilingual ASR
Building ASR models across many languages is a challenging multi-task learning problem due to large variations and heavily unbalanced data.
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition
We introduce Lookup-Table Language Models (LookupLM), a method for scaling up the size of RNN language models with only a constant increase in the floating point operations, by increasing the expressivity of the embedding table.
Learning Word-Level Confidence For Subword End-to-End ASR
We study the problem of word-level confidence estimation in subword-based end-to-end (E2E) models for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Transformer Based Deliberation for Two-Pass Speech Recognition
In this work, we explore using transformer layers instead of long-short term memory (LSTM) layers for deliberation rescoring.
Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging
End-to-end models that condition the output label sequence on all previously predicted labels have emerged as popular alternatives to conventional systems for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Better and Faster End-to-End Model for Streaming ASR
To address this, we explore replacing the LSTM layers in the encoder of our E2E model with Conformer layers [4], which has shown good improvements for ASR.
Audio and Speech Processing Sound
Cascaded encoders for unifying streaming and non-streaming ASR
The proposed model consists of streaming and non-streaming encoders.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multitask Training with Text Data for End-to-End Speech Recognition
We propose a multitask training method for attention-based end-to-end speech recognition models.
FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization
FastEmit also improves streaming ASR accuracy from 4. 4%/8. 9% to 3. 1%/7. 5% WER, meanwhile reduces 90th percentile latency from 210 ms to only 30 ms on LibriSpeech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible, while full-context ASR waits for the completion of a full speech utterance before emitting completed hypotheses.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpus
End-to-end (E2E) automatic speech recognition (ASR) systems lack the distinct language model (LM) component that characterizes traditional speech systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improving Proper Noun Recognition in End-to-End ASR By Customization of the MWER Loss Criterion
Proper nouns present a challenge for end-to-end (E2E) automatic speech recognition (ASR) systems in that a particular name may appear only rarely during training, and may have a pronunciation similar to that of a more common word.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
On a long-form YouTube test set, when the nonstreaming RNN-T model is trained with shorter segments of data, the proposed combination improves word error rate (WER) from 22. 3% to 14. 8%; when the streaming RNN-T model trained on short Search queries, the proposed techniques improve WER on the YouTube set from 67. 0% to 25. 3%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Towards Fast and Accurate Streaming End-to-End ASR
RNN-T EP+LAS, together with MWER training brings in 18. 7% relative WER reduction and 160ms 90-percentile latency reductions compared to the original proposed RNN-T EP model.
Audio and Speech Processing
A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency
Thus far, end-to-end (E2E) models have not been shown to outperform state-of-the-art conventional models with respect to both quality, i. e., word error rate (WER), and latency, i. e., the time the hypothesis is finalized after the user stops speaking.
Deliberation Model Based Two-Pass End-to-End Speech Recognition
End-to-end (E2E) models have made rapid progress in automatic speech recognition (ASR) and perform competitively relative to conventional models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Recognizing long-form speech using streaming end-to-end models
In this work, we examine the ability of E2E models to generalize to unseen domains, where we find that models trained on short utterances fail to generalize to long-form speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model
Multilingual end-to-end (E2E) models have shown great promise in expansion of automatic speech recognition (ASR) coverage of the world's languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Two-Pass End-to-End Speech Recognition
However, this model still lags behind a large state-of-the-art conventional model in quality [2].
Improving Performance of End-to-End ASR on Numeric Sequences
This out-of-vocabulary (OOV) issue is addressed in conventional ASR systems by training part of the model on spoken domain utterances (e. g.
Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models
Contextual automatic speech recognition, i. e., biasing recognition towards a given context (e. g. user's playlists, or contacts), is challenging in end-to-end (E2E) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A spelling correction model for end-to-end speech recognition
Attention-based sequence-to-sequence models for speech recognition jointly train an acoustic model, language model (LM), and alignment mechanism using a single neural network and require only parallel audio-text pairs.
Streaming End-to-end Speech Recognition For Mobile Devices
End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition.
Contextual Speech Recognition with Difficult Negative Training Examples
Improving the representation of contextual information is key to unlocking the potential of end-to-end (E2E) automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Deep context: end-to-end contextual speech recognition
Our approach, which we re- fer to as Contextual Listen, Attend and Spell (CLAS) jointly- optimizes the ASR components along with embeddings of the context n-grams.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition
In this paper, we compare a suite of past methods and some of our own proposed methods for using unpaired text data to improve encoder-decoder models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
An analysis of incorporating an external language model into a sequence-to-sequence model
Attention-based sequence-to-sequence models for automatic speech recognition jointly train an acoustic model, language model, and alignment mechanism.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models
However, there has been little previous work comparing phoneme-based versus grapheme-based sub-word units in the end-to-end modeling framework, to determine whether the gains from such approaches are primarily due to the new probabilistic model, or from the joint learning of the various components with grapheme-based units.
Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network.
Improving the Performance of Online Neural Transducer Models
Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (LAS).
Multilingual Speech Recognition With A Single End-To-End Model
Training a conventional automatic speech recognition (ASR) system to support multiple languages is challenging because the sub-word unit, lexicon and word inventories are typically language specific.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Learning Compact Recurrent Neural Networks
Recurrent neural networks (RNNs), including long short-term memory (LSTM) RNNs, have produced state-of-the-art results on a variety of speech recognition tasks.
Structured Transforms for Small-Footprint Deep Learning
We consider the task of building compact deep learning pipelines suitable for deployment on storage and power constrained mobile devices.
Improvements to deep convolutional neural networks for LVCSR
We find that with these improvements, particularly with fMLLR and dropout, we are able to achieve an additional 2-3% relative improvement in WER on a 50-hour Broadcast News task over our previous best CNN baseline.
Accelerating Hessian-free optimization for deep neural networks by implicit preconditioning and sampling
This study aims at speeding up Hessian-free training, both by means of decreasing the amount of data used for training, as well as through reduction of the number of Krylov subspace solver iterations used for implicit estimation of the Hessian.
