Search Results for author:
Found 33 papers, 5 papers with code
NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts
On zero-shot evaluation, NeKo outperforms GPT-3. 5 and Claude-Opus with $15. 5$% to $27. 6$% relative WER reduction in the Hyporadise benchmark.
Anticipating Future with Large Language Model for Simultaneous Machine Translation
Simultaneous machine translation (SMT) takes streaming input utterances and incrementally produces target text.
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning
More recent studies have extended this to multi-turn conversations, though they often require complex, multi-stage supervised fine-tuning (SFT) with diverse data.
Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
Recent end-to-end speech language models (SLMs) have expanded upon the capabilities of large language models (LLMs) by incorporating pre-trained speech models.
EMMeTT: Efficient Multimodal Machine Translation Training
This work focuses on neural machine translation (NMT) and proposes a joint multimodal training regime of Speech-LLM to include automatic speech translation (AST).
META-CAT: Speaker-Informed Speech Embeddings via Meta Information Concatenation for Multi-talker ASR
We propose a novel end-to-end multi-talker automatic speech recognition (ASR) framework that enables both multi-speaker (MS) ASR and target-speaker (TS) ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Chain-of-Thought Prompting for Speech Translation
Building on the success of text-based LLMs, recent research has adapted these models to use speech embeddings for prompting, resulting in Speech-LLM models that exhibit strong performance in automatic speech recognition (ASR) and automatic speech translation (AST).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Sortformer: Seamless Integration of Speaker Diarization and ASR by Bridging Timestamps and Tokens
We demonstrate that combining Sort Loss and PIL achieves performance competitive with state-of-the-art end-to-end diarization models trained exclusively with PIL.
Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translation
Additionally, training on longer audio segments increases the overall model accuracy across speech recognition and translation benchmarks.
Resource-Efficient Adaptation of Speech Foundation Models for Multi-Speaker ASR
Speech foundation models have achieved state-of-the-art (SoTA) performance across various tasks, such as automatic speech recognition (ASR) in hundreds of languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
Large Language Models (LLMs) rely on instruction samples for alignment, but creating these datasets poses challenges, particularly in expert-dependent tasks like coding, which can be cost-prohibitive.
Schrödinger Bridge for Generative Speech Enhancement
This paper proposes a generative speech enhancement model based on Schr\"odinger bridge (SB).
Ranked #4 on
Speech Enhancement
on EARS-WHAM
Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representations
Discrete speech representations have garnered recent attention for their efficacy in training transformer-based models for various speech-related tasks such as automatic speech recognition (ASR), translation, speaker verification, and joint speech-text foundational models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5
We propose BESTOW architecture to bring the BESt features from TwO Worlds into a single model that is highly efficient and has strong multitask capabilities.
Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data.
Instruction Data Generation and Unsupervised Adaptation for Speech Language Models
In this paper, we propose three methods for generating synthetic samples to train and evaluate multimodal large language models capable of processing both text and speech inputs.
Flexible Multichannel Speech Enhancement for Noise-Robust Frontend
This paper proposes a flexible multichannel speech enhancement system with the main goal of improving robustness of automatic speech recognition (ASR) in noisy conditions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer
Humans are adept at leveraging visual cues from lip movements for recognizing speech in adverse listening conditions.
Audio-Visual Speech Recognition
Robust Speech Recognition
+2
Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition
We also showed that training a model with multiple latencies can achieve better accuracy than single latency models while it enables us to support multiple latencies with a single model.
Property-Aware Multi-Speaker Data Simulation: A Probabilistic Modelling Technique for Synthetic Data Generation
This capability offers a tailored training environment for developing neural models suited for speaker diarization and voice activity detection.
The CHiME-7 Challenge: System Description and Performance of NeMo Team's DASR System
We present the NVIDIA NeMo team's multi-channel speech recognition system for the 7th CHiME Challenge Distant Automatic Speech Recognition (DASR) Task, focusing on the development of a multi-channel, multi-speaker speech recognition system tailored to transcribe speech from distributed microphones and microphone arrays.
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation
We present a novel Speech Augmented Language Model (SALM) with {\em multitask} and {\em in-context} learning capabilities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition
Discrete audio representation, aka audio tokenization, has seen renewed interest driven by its potential to facilitate the application of text language modeling approaches in audio domain.
Investigating End-to-End ASR Architectures for Long Form Audio Transcription
This paper presents an overview and evaluation of some of the end-to-end ASR models on long-form audios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach
In addition, these findings point to the potential of using LLMs to improve speaker diarization and other speech processing tasks by capturing semantic and contextual cues.
Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Filling
We study speech intent classification and slot filling (SICSF) by proposing to use an encoder pretrained on speech recognition (ASR) to initialize an end-to-end (E2E) Conformer-Transformer model, which achieves the new state-of-the-art results on the SLURP dataset, with 90. 14% intent accuracy and 82. 27% SLURP-F1.
Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition
Conformer-based models have become the dominant end-to-end architecture for speech processing tasks.
Ranked #1 on
Speech Recognition
on Common Voice English
(using extra training data)
A Compact End-to-End Model with Local and Global Context for Spoken Language Identification
We introduce TitaNet-LID, a compact end-to-end neural network for Spoken Language Identification (LID) that is based on the ContextNet architecture.
Multi-scale Speaker Diarization with Dynamic Scale Weighting
First, we use multi-scale clustering as an initialization to estimate the number of speakers and obtain the average speaker representation vector for each speaker and each scale.
CarneliNet: Neural Mixture Model for Automatic Speech Recognition
The basic idea is to introduce a parallel mixture of shallow networks instead of a very deep network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
In the English speech-to-text (STT) machine learning task, acoustic models are conventionally trained on uncased Latin characters, and any necessary orthography (such as capitalization, punctuation, and denormalization of non-standard words) is imputed by separate post-processing models.
Ranked #3 on
Speech Recognition
on SPGISpeech
Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
We propose Citrinet - a new end-to-end convolutional Connectionist Temporal Classification (CTC) based automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
