Search Results for author:
Found 37 papers, 2 papers with code
Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR
Meanwhile, the heterogeneous nature and imbalanced data abundance of different languages may cause performance degradation, leading to asynchronous peak performance for different languages during training, especially on tail ones.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Controlled Decoding from Language Models
KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes.
Language Modelling
Multi-Objective Reinforcement Learning
+1
Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR
Conformer models maintain a large number of internal states, the vast majority of which are associated with self-attention layers.
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
UML: A Universal Monolingual Output Layer for Multilingual ASR
Consequently, the UML enables to switch in the interpretation of each output node depending on the language of the input speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Efficient Domain Adaptation for Speech Foundation Models
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition
In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion
Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Modular Hybrid Autoregressive Transducer
In this work, we propose a modular hybrid autoregressive transducer (MHAT) that has structurally separated label and blank decoders to predict label and blank distributions, respectively, along with a shared acoustic encoder.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR
Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification
Language identification is critical for many downstream tasks in automatic speech recognition (ASR), and is beneficial to integrate into multilingual end-to-end ASR as an additional task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Language Agnostic Multilingual Streaming On-Device ASR System
On-device end-to-end (E2E) models have shown improvements over a conventional model on English Voice Search tasks in both quality and latency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Turn-Taking Prediction for Natural Conversational Speech
This makes doing speech recognition with conversational speech, including one with multiple queries, a challenging task.
Streaming Intended Query Detection using E2E Modeling for Continued Conversation
In voice-enabled applications, a predetermined hotword isusually used to activate a device in order to attend to the query. However, speaking queries followed by a hotword each timeintroduces a cognitive burden in continued conversations.
Improving Deliberation by Text-Only and Semi-Supervised Training
Text-only and semi-supervised training based on audio-only data has gained popularity recently due to the wide availability of unlabeled text and speech data.
Improving Rare Word Recognition with LM-aware MWER Training
Language models (LMs) significantly improve the recognition accuracy of end-to-end (E2E) models on words rarely seen during training, when used in either the shallow fusion or the rescoring setups.
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Pseudo Label Is Better Than Human Label
State-of-the-art automatic speech recognition (ASR) systems are trained with tens of thousands of hours of labeled speech data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition
We down-select a large corpus of web search queries by a factor of 53x and achieve better LM perplexities than without down-selection.
Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition
Our experiments show that for both losses, the WER on long-form speech reduces substantially as the training utterance length increases.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition
Fast contextual adaptation has shown to be effective in improving Automatic Speech Recognition (ASR) of rare words and when combined with an on-device personalized training, it can yield an even better recognition result.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learning
Self- and semi-supervised learning methods have been actively investigated to reduce labeled training data or enhance the model performance.
Incremental Layer-wise Self-Supervised Learning for Efficient Speech Domain Adaptation On Device
These models are typically trained on the server using transcribed speech data.
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition
We introduce Lookup-Table Language Models (LookupLM), a method for scaling up the size of RNN language models with only a constant increase in the floating point operations, by increasing the expressivity of the embedding table.
Transformer Based Deliberation for Two-Pass Speech Recognition
In this work, we explore using transformer layers instead of long-short term memory (LSTM) layers for deliberation rescoring.
Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging
End-to-end models that condition the output label sequence on all previously predicted labels have emerged as popular alternatives to conventional systems for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Better and Faster End-to-End Model for Streaming ASR
To address this, we explore replacing the LSTM layers in the encoder of our E2E model with Conformer layers [4], which has shown good improvements for ASR.
Audio and Speech Processing Sound
Cascaded encoders for unifying streaming and non-streaming ASR
The proposed model consists of streaming and non-streaming encoders.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition
For various speech-related tasks, confidence scores from a speech recogniser are a useful measure to assess the quality of transcriptions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards Fast and Accurate Streaming End-to-End ASR
RNN-T EP+LAS, together with MWER training brings in 18. 7% relative WER reduction and 160ms 90-percentile latency reductions compared to the original proposed RNN-T EP model.
Audio and Speech Processing
A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency
Thus far, end-to-end (E2E) models have not been shown to outperform state-of-the-art conventional models with respect to both quality, i. e., word error rate (WER), and latency, i. e., the time the hypothesis is finalized after the user stops speaking.
Recognizing long-form speech using streaming end-to-end models
In this work, we examine the ability of E2E models to generalize to unseen domains, where we find that models trained on short utterances fail to generalize to long-form speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Two-Pass End-to-End Speech Recognition
However, this model still lags behind a large state-of-the-art conventional model in quality [2].
Toward domain-invariant speech recognition via large scale training
More importantly, such models generalize better to unseen conditions and allow for rapid adaptation -- we show that by using as little as 10 hours of data from a new domain, an adapted domain-invariant model can match performance of a domain-specific model trained from scratch using 70 times as much data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
