Search Results for author:
Found 27 papers, 2 papers with code
Chain-of-Thought Prompting for Speech Translation
Building on the success of text-based LLMs, recent research has adapted these models to use speech embeddings for prompting, resulting in Speech-LLM models that exhibit strong performance in automatic speech recognition (ASR) and automatic speech translation (AST).
Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model.
BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5
We propose BESTOW architecture to bring the BESt features from TwO Worlds into a single model that is highly efficient and has strong multitask capabilities.
Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data.
DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
Recent speech language models (SLMs) typically incorporate pre-trained speech models to extend the capabilities from large language models (LLMs).
Instruction Data Generation and Unsupervised Adaptation for Speech Language Models
In this paper, we propose three methods for generating synthetic samples to train and evaluate multimodal large language models capable of processing both text and speech inputs.
Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition
This paper proposes Transducers with Pronunciation-aware Embeddings (PET).
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Leveraging the rich linguistic knowledge and strong reasoning abilities of LLMs, our new paradigm can integrate the rich information in N-best candidates to generate a higher-quality translation result.
Ranked #1 on
Machine Translation
on FLoRes-200
High-precision Voice Search Query Correction via Retrievable Speech-text Embedings
In this paper, we eliminate the hypothesis-audio mismatch problem by querying the correction database directly using embeddings derived from the utterance audio; the embeddings of the utterance audio and candidate corrections are produced by multimodal speech-text embedding networks trained to place the embedding of the audio of an utterance and the embedding of its corresponding textual transcript close together.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation
We present a novel Speech Augmented Language Model (SALM) with {\em multitask} and {\em in-context} learning capabilities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Using Text Injection to Improve Recognition of Personal Identifiers in Speech
We use text-injection to improve the recognition of PII categories by including fake textual substitutes of PII categories in the training data using a text injection method.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Understanding Shared Speech-Text Representations
Recently, a number of approaches to train speech models by incorpo-rating text into end-to-end models have been developed, with Mae-stro advancing state-of-the-art automatic speech recognition (ASR)and Speech Translation (ST) performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Accelerating RNN-T Training and Inference Using CTC guidance
We propose a novel method to accelerate training and inference process of recurrent neural network transducer (RNN-T) based on the guidance from a co-trained connectionist temporal classification (CTC) model.
Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech
This paper proposes Virtuoso, a massively multilingual speech-text joint semi-supervised learning framework for text-to-speech synthesis (TTS) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASR
First, we show that by combining speech representations with byte-level text representations and use of language embeddings, we can dramatically reduce the Character Error Rate (CER) on languages with no supervised speech from 64. 8\% to 30. 8\%, a relative reduction of 53\%.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data
However, there are not enough data sets for accented speech, and for the ones that are already available, more training approaches need to be explored to improve the quality of accented speech recognition.
MAESTRO: Matched Speech Text Representations through Modality Matching
Self-supervised learning from speech signals aims to learn the latent structure inherent in the signal, while self-supervised learning from text attempts to capture lexical information.
Unsupervised Data Selection via Discrete Speech Representation for ASR
Self-supervised learning of speech representations has achieved impressive results in improving automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Injecting Text in Self-Supervised Speech Pretraining
The proposed method, tts4pretrain complements the power of contrastive learning in self-supervision with linguistic/lexical representations derived from synthesized speech, effectively learning from untranscribed speech and unspoken text.
End-to-end contextual speech recognition using class language models and a token passing decoder
In this work, we focus on contextual speech recognition, which is particularly challenging for E2E models because it introduces significant mismatch between training and test data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Sequence Discriminative Training for Deep Learning based Acoustic Keyword Spotting
The few studies on sequence discriminative training for KWS are limited for fixed vocabulary or LVCSR based methods and have not been compared to the state-of-the-art deep learning based KWS approaches.
Linguistic Search Optimization for Deep Learning Based LVCSR
Recent advances in deep learning based large vocabulary con- tinuous speech recognition (LVCSR) invoke growing demands in large scale speech transcription.
A GPU-based WFST Decoder with Exact Lattice Generation
We describe initial work on an extension of the Kaldi toolkit that supports weighted finite-state transducer (WFST) decoding on Graphics Processing Units (GPUs).
On Modular Training of Neural Acoustics-to-Word Model for LVCSR
Finally, modules are integrated into an acousticsto-word model (A2W) and jointly optimized using acoustic data to retain the advantage of sequence modeling.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
We propose to advance the current state of the art by imposing a modular structure on the neural network, applying a progressive pretraining regimen, and improving the objective function with transfer learning and a discriminative training criterion.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
