Search Results for author:
Found 31 papers, 4 papers with code
Effective internal language model training and fusion for factorized transducer model
However, even with the adoption of factorized transducer models, limited improvement has been observed compared to shallow fusion.
AGADIR: Towards Array-Geometry Agnostic Directional Speech Recognition
Wearable devices like smart glasses are approaching the compute capability to seamlessly generate real-time closed captions for live conversations.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data.
End-to-End Speech Recognition Contextualization with Large Language Models
Overall, we demonstrate that by only adding a handful number of trainable parameters via adapters, we can unlock contextualized speech recognition capability for the pretrained LLM while keeping the same text-only input functionality.
Prompting Large Language Models with Speech Recognition Abilities
Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings.
Abstractive Text Summarization
Automatic Speech Recognition
+3
SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervision
Furthermore, when combined with large-scale pseudo-labeled audio-visual data SynthVSR yields a new state-of-the-art VSR WER of 16. 9% using publicly available data only, surpassing the recent state-of-the-art approaches trained with 29 times more non-public machine-transcribed video data (90, 000 hours).
Streaming Audio-Visual Speech Recognition with Alignment Regularization
In this work, we propose a streaming AV-ASR system based on a hybrid connectionist temporal classification (CTC)/attention neural network architecture.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+5
An Investigation of Monotonic Transducers for Large-Scale Automatic Speech Recognition
The two most popular loss functions for streaming end-to-end automatic speech recognition (ASR) are RNN-Transducer (RNN-T) and connectionist temporal classification (CTC).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Scaling ASR Improves Zero and Few Shot Learning
With 4. 5 million hours of English speech from 10 different sources across 120 countries and models of up to 10 billion parameters, we explore the frontiers of scale for automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Evaluating User Perception of Speech Recognition System Quality with Semantic Distance Metric
Measuring automatic speech recognition (ASR) system quality is critical for creating user-satisfying voice-driven applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Do sound event representations generalize to other audio tasks? A case study in audio transfer learning
We also provide insights into the attributes of sound event representations that enable such efficient information transfer.
Dissecting User-Perceived Latency of On-Device E2E Speech Recognition
As speech-enabled devices such as smartphones and smart speakers become increasingly ubiquitous, there is growing interest in building automatic speech recognition (ASR) systems that can run directly on-device; end-to-end (E2E) speech recognition models such as recurrent neural network transducers and their variants have recently emerged as prime candidates for this task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding
We define SemDist as the distance between a reference and hypothesis pair in a sentence-level embedding space.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+14
Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
DET gets similar accuracy as a baseline model with better latency on a large in-house data set by assigning a lightweight encoder for the beginning part of one utterance and a full-size encoder for the rest.
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion
How to leverage dynamic contextual information in end-to-end speech recognition has remained an active research area.
Contrastive Semi-supervised Learning for ASR
Pseudo-labeling is the most adopted method for pre-training automatic speech recognition (ASR) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Memory-efficient Speech Recognition on Smart Devices
Recurrent transducer models have emerged as a promising solution for speech recognition on the current and next generation smart devices.
Deep Shallow Fusion for RNN-T Personalization
End-to-end models in general, and Recurrent Neural Network Transducer (RNN-T) in particular, have gained significant traction in the automatic speech recognition community in the last few years due to their simplicity, compactness, and excellent performance on generic transcription tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Alignment Restricted Streaming Recurrent Neural Network Transducer
There is a growing interest in the speech community in developing Recurrent Neural Network Transducer (RNN-T) models for automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer
Recurrent Neural Network Transducer (RNN-T), like most end-to-end speech recognition model architectures, has an implicit neural network language model (NNLM) and cannot easily leverage unpaired text data during training.
Weak-Attention Suppression For Transformer Based Speech Recognition
Transformers, originally proposed for natural language processing (NLP) tasks, have recently achieved great success in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Large scale weakly and semi-supervised learning for low-resource video ASR
Many semi- and weakly-supervised approaches have been investigated for overcoming the labeling cost of building high quality speech recognition systems.
Libri-Light: A Benchmark for ASR with Limited or No Supervision
Additionally, we provide baseline systems and evaluation metrics working under three settings: (1) the zero resource/unsupervised setting (ABX), (2) the semi-supervised setting (PER, CER) and (3) the distant supervision setting (WER).
Ranked #1 on
Speech Recognition
on Libri-Light test-other
(ABX-within metric)
RNN-T For Latency Controlled ASR With Improved Beam Search
Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic speech recognition (ASR) blend the individual components of a traditional hybrid ASR systems (acoustic model, language model, punctuation model, inverse text normalization) into one single model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
We explore options to use Transformer networks in neural transducer for end-to-end speech recognition.
G2G: TTS-Driven Pronunciation Learning for Graphemic Hybrid ASR
Grapheme-based acoustic modeling has recently been shown to outperform phoneme-based approaches in both hybrid and end-to-end automatic speech recognition (ASR), even on non-phonemic languages like English.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Transformer-based Acoustic Modeling for Hybrid Speech Recognition
We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition.
Ranked #23 on
Speech Recognition
on LibriSpeech test-other
(using extra training data)
End-to-end contextual speech recognition using class language models and a token passing decoder
In this work, we focus on contextual speech recognition, which is particularly challenging for E2E models because it introduces significant mismatch between training and test data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Towards end-to-end spoken language understanding
Spoken language understanding system is traditionally designed as a pipeline of a number of components.
Natural Language Understanding
Spoken Language Understanding
