Search Results for author:
Found 43 papers, 2 papers with code
Faster Speech-LLaMA Inference with Multi-token Prediction
Large language models (LLMs) have become proficient at solving a wide variety of tasks, including those involving multi-modal inputs.
Towards measuring fairness in speech recognition: Fair-Speech dataset
The current public datasets for speech recognition (ASR) tend not to focus specifically on the fairness aspect, such as performance across different demographic groups.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Multi-task Language Understanding
on MMLU
Token-Weighted RNN-T for Learning from Flawed Data
ASR models are commonly trained with the cross-entropy criterion to increase the probability of a target token sequence.
Effective internal language model training and fusion for factorized transducer model
However, even with the adoption of factorized transducer models, limited improvement has been observed compared to shallow fusion.
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data.
Correction Focused Language Model Training for Speech Recognition
Language models (LMs) have been commonly adopted to boost the performance of automatic speech recognition (ASR) particularly in domain adaptation tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemble
Recent research has shown that language models have a tendency to memorize rare or unique token sequences in the training corpus.
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model
In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
End-to-End Speech Recognition Contextualization with Large Language Models
Overall, we demonstrate that by only adding a handful number of trainable parameters via adapters, we can unlock contextualized speech recognition capability for the pretrained LLM while keeping the same text-only input functionality.
Augmenting text for spoken language understanding with Large Language Models
Using the generated text with JAT and TTS for spoken semantic parsing improves EM on STOP by 1. 4% and 2. 6% absolute for existing and new domains respectively.
Recovering from Privacy-Preserving Masking with Large Language Models
To effectively perform adaptation, textual data of users is typically stored on servers or their local devices, where downstream natural language processing (NLP) models can be directly trained using such in-domain data.
TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
Results demonstrate that our TODM Supernet either matches or surpasses the performance of manually tuned models by up to a relative of 3% better in word error rate (WER), while efficiently keeping the cost of training many models at a small constant.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Contextual Biasing of Named-Entities with Large Language Models
We propose to leverage prompts for a LLM without fine tuning during rescoring which incorporate a biasing list and few-shot examples to serve as additional information when calculating the score for the hypothesis.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding
End-to-end (E2E) spoken language understanding (SLU) systems that generate a semantic parse from speech have become more promising recently.
Prompting Large Language Models with Speech Recognition Abilities
Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings.
Abstractive Text Summarization
Automatic Speech Recognition
+3
Towards Selection of Text-to-speech Data to Augment ASR Training
This paper presents a method for selecting appropriate synthetic speech samples from a given large text-to-speech (TTS) dataset as supplementary training data for an automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multi-Head State Space Model for Speech Recognition
State space models (SSMs) have recently shown promising results on small-scale sequence and language modelling tasks, rivalling and outperforming many attention-based approaches.
Ranked #8 on
Speech Recognition
on LibriSpeech test-clean
Improving Fast-slow Encoder based Transducer with Streaming Deliberation
Experiments on Librispeech and in-house data show relative WER reductions (WERRs) from 3% to 5% with a slight increase in model size and negligible extra token emission latency compared with fast-slow encoder based transducer.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities
Later, we use our optimal tokenization strategy to train multiple embedding and output model to further improve our result.
Factorized Blank Thresholding for Improved Runtime Efficiency of Neural Transducers
We show how factoring the RNN-T's output distribution can significantly reduce the computation cost and power consumption for on-device ASR inference with no loss in accuracy.
Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition
In this work, we present our Joint Audio/Text training method for Transformer Rescorer, to leverage unpaired text-only data which is relatively cheaper than paired audio-text data.
Anchored Speech Recognition with Neural Transducers
In this paper, we investigate anchored speech recognition to make neural transducers robust to background speech.
Learning ASR pathways: A sparse multilingual ASR model
Neural network pruning compresses automatic speech recognition (ASR) models effectively.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Learning a Dual-Mode Speech Recognition Model via Self-Pruning
There is growing interest in unifying the streaming and full-context automatic speech recognition (ASR) networks into a single end-to-end ASR model to simplify the model training and deployment for both use cases.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Deliberation Model for On-Device Spoken Language Understanding
We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU), where a streaming automatic speech recognition (ASR) model produces the first-pass hypothesis and a second-pass natural language understanding (NLU) component generates the semantic parse by conditioning on both ASR's text and audio embeddings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Federated Domain Adaptation for ASR with Full Self-Supervision
Cross-device federated learning (FL) protects user privacy by collaboratively training a model on user devices, therefore eliminating the need for collecting, storing, and manually labeling user data.
Streaming parallel transducer beam search with fast-slow cascaded encoders
Streaming ASR with strict latency constraints is required in many speech recognition applications.
Neural-FST Class Language Model for End-to-End Speech Recognition
We propose Neural-FST Class Language Model (NFCLM) for end-to-end speech recognition, a novel method that combines neural network language models (NNLMs) and finite state transducers (FSTs) in a mathematically consistent framework.
Scaling ASR Improves Zero and Few Shot Learning
With 4. 5 million hours of English speech from 10 different sources across 120 countries and models of up to 10 billion parameters, we explore the frontiers of scale for automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Omni-sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR via Supernet
From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Evaluating User Perception of Speech Recognition System Quality with Semantic Distance Metric
Measuring automatic speech recognition (ASR) system quality is critical for creating user-satisfying voice-driven applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution
This paper improves the streaming transformer transducer for speech recognition by using non-causal convolution.
Transferring Voice Knowledge for Acoustic Event Detection: An Empirical Study
Detection of common events and scenes from audio is useful for extracting and understanding human contexts in daily life.
Noisy Training Improves E2E ASR for the Edge
We apply noisy training to improve both dense and sparse state-of-the-art Emformer models and observe consistent WER reduction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Collaborative Training of Acoustic Encoders for Speech Recognition
On-device speech recognition requires training models of different sizes for deploying on devices with various computational budgets.
Dissecting User-Perceived Latency of On-Device E2E Speech Recognition
As speech-enabled devices such as smartphones and smart speakers become increasingly ubiquitous, there is growing interest in building automatic speech recognition (ASR) systems that can run directly on-device; end-to-end (E2E) speech recognition models such as recurrent neural network transducers and their variants have recently emerged as prime candidates for this task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion
How to leverage dynamic contextual information in end-to-end speech recognition has remained an active research area.
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding
We define SemDist as the distance between a reference and hypothesis pair in a sentence-level embedding space.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+14
Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
DET gets similar accuracy as a baseline model with better latency on a large in-house data set by assigning a lightweight encoder for the beginning part of one utterance and a full-size encoder for the rest.
SapAugment: Learning A Sample Adaptive Policy for Data Augmentation
Our policy adapts the augmentation parameters based on the training loss of the data samples.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Bandwidth Embeddings for Mixed-bandwidth Speech Recognition
In this paper, we tackle the problem of handling narrowband and wideband speech by building a single acoustic model (AM), also called mixed bandwidth AM.
