Search Results for author:
Found 33 papers, 3 papers with code
CJST: CTC Compressor based Joint Speech and Text Training for Decoder-Only ASR
CTC compressor can be an effective approach to integrate audio encoders to decoder-only models, which has gained growing interest for different speech applications.
Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech
As speech becomes an increasingly common modality for interacting with large language models (LLMs), it is becoming desirable to develop systems where LLMs can take into account users' emotions or speaking styles when providing their responses.
Efficient Streaming LLM for Speech Recognition
Recent works have shown that prompting large language models with audio encodings can unlock speech recognition capabilities.
M-BEST-RQ: A Multi-Channel Speech Foundation Model for Smart Glasses
For the conversational ASR task in particular, using only 8 hours of labeled speech, our model outperforms a supervised ASR baseline that is trained on 2000 hours of labeled data, which demonstrates the effectiveness of our approach.
Faster Speech-LLaMA Inference with Multi-token Prediction
Large language models (LLMs) have become proficient at solving a wide variety of tasks, including those involving multi-modal inputs.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Towards scalable efficient on-device ASR with transfer learning
Multilingual pretraining for transfer learning significantly boosts the robustness of low-resource monolingual ASR models.
Effective internal language model training and fusion for factorized transducer model
However, even with the adoption of factorized transducer models, limited improvement has been observed compared to shallow fusion.
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data.
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model
In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
Results demonstrate that our TODM Supernet either matches or surpasses the performance of manually tuned models by up to a relative of 3% better in word error rate (WER), while efficiently keeping the cost of training many models at a small constant.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Prompting Large Language Models with Speech Recognition Abilities
Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings.
Abstractive Text Summarization
Automatic Speech Recognition
+3
Towards Selection of Text-to-speech Data to Augment ASR Training
This paper presents a method for selecting appropriate synthetic speech samples from a given large text-to-speech (TTS) dataset as supplementary training data for an automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multi-Head State Space Model for Speech Recognition
State space models (SSMs) have recently shown promising results on small-scale sequence and language modelling tasks, rivalling and outperforming many attention-based approaches.
Ranked #11 on
Speech Recognition
on LibriSpeech test-clean
Improving Fast-slow Encoder based Transducer with Streaming Deliberation
Experiments on Librispeech and in-house data show relative WER reductions (WERRs) from 3% to 5% with a slight increase in model size and negligible extra token emission latency compared with fast-slow encoder based transducer.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Anchored Speech Recognition with Neural Transducers
In this paper, we investigate anchored speech recognition to make neural transducers robust to background speech.
An Investigation of Monotonic Transducers for Large-Scale Automatic Speech Recognition
The two most popular loss functions for streaming end-to-end automatic speech recognition (ASR) are RNN-Transducer (RNN-T) and connectionist temporal classification (CTC).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Federated Domain Adaptation for ASR with Full Self-Supervision
Cross-device federated learning (FL) protects user privacy by collaboratively training a model on user devices, therefore eliminating the need for collecting, storing, and manually labeling user data.
Streaming parallel transducer beam search with fast-slow cascaded encoders
Streaming ASR with strict latency constraints is required in many speech recognition applications.
TorchAudio: Building Blocks for Audio and Speech Processing
This document describes version 0. 10 of TorchAudio: building blocks for machine learning applications in the audio and speech processing domain.
Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution
This paper improves the streaming transformer transducer for speech recognition by using non-causal convolution.
Dissecting User-Perceived Latency of On-Device E2E Speech Recognition
As speech-enabled devices such as smartphones and smart speakers become increasingly ubiquitous, there is growing interest in building automatic speech recognition (ASR) systems that can run directly on-device; end-to-end (E2E) speech recognition models such as recurrent neural network transducers and their variants have recently emerged as prime candidates for this task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion
How to leverage dynamic contextual information in end-to-end speech recognition has remained an active research area.
Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
DET gets similar accuracy as a baseline model with better latency on a large in-house data set by assigning a lightweight encoder for the beginning part of one utterance and a full-size encoder for the rest.
Memory-efficient Speech Recognition on Smart Devices
Recurrent transducer models have emerged as a promising solution for speech recognition on the current and next generation smart devices.
Deep Shallow Fusion for RNN-T Personalization
End-to-end models in general, and Recurrent Neural Network Transducer (RNN-T) in particular, have gained significant traction in the automatic speech recognition community in the last few years due to their simplicity, compactness, and excellent performance on generic transcription tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Alignment Restricted Streaming Recurrent Neural Network Transducer
There is a growing interest in the speech community in developing Recurrent Neural Network Transducer (RNN-T) models for automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer
Recurrent Neural Network Transducer (RNN-T), like most end-to-end speech recognition model architectures, has an implicit neural network language model (NNLM) and cannot easily leverage unpaired text data during training.
Contextual RNN-T For Open Domain ASR
By using an attention model and a biasing model to leverage the contextual metadata that accompanies a video, we observe a relative improvement of about 16% in Word Error Rate on Named Entities (WER-NE) for videos with related metadata.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
RNN-T For Latency Controlled ASR With Improved Beam Search
Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic speech recognition (ASR) blend the individual components of a traditional hybrid ASR systems (acoustic model, language model, punctuation model, inverse text normalization) into one single model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
We explore options to use Transformer networks in neural transducer for end-to-end speech recognition.
Transformer-based Acoustic Modeling for Hybrid Speech Recognition
We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition.
Ranked #28 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
