Search Results for author:
Found 17 papers, 8 papers with code
SSR: Alignment-Aware Modality Connector for Speech Language Models
Fusing speech into pre-trained language model (SpeechLM) usually suffers from inefficient encoding of long-form speech and catastrophic forgetting of pre-trained text modality.
Seamless: Multilingual Expressive and Streaming Speech Translation
In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion.
automatic-speech-translation
Multimodal Machine Translation
+2
Efficient Monotonic Multihead Attention
We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation.
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
Ranked #1 on
Speech-to-Speech Translation
on CVSS
(using extra training data)
Automatic Speech Recognition
Speech-to-Speech Translation
+4
Scaling Speech Technology to 1,000+ Languages
Expanding the language coverage of speech technology has the potential to improve access to information for many more people.
Efficient Speech Representation Learning with Low-Bit Quantization
With the development of hardware for machine learning, newer models often come at the cost of both increased sizes and computational complexity.
Continual Learning for On-Device Speech Recognition using Disentangled Conformers
Additionally, current speech recognition models and continual learning algorithms are not optimized to be compute-efficient.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speech-to-Speech Translation For A Real-world Unwritten Language
We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
STOP: A dataset for Spoken Task Oriented Semantic Parsing
Furthermore, in addition to the human-recorded audio, we are releasing a TTS-generated version to benchmark the performance for low-resource domain adaptation of end-to-end SLU systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Deliberation Model for On-Device Spoken Language Understanding
We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU), where a streaming automatic speech recognition (ASR) model produces the first-pass hypothesis and a second-pass natural language understanding (NLU) component generates the semantic parse by conditioning on both ASR's text and audio embeddings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Generative Spoken Dialogue Language Modeling
We introduce dGSLM, the first "textless" model able to generate audio samples of naturalistic spoken dialogues.
textless-lib: a Library for Textless Spoken Language Processing
Textless spoken language processing research aims to extend the applicability of standard NLP toolset onto spoken language and languages with few or no textual resources.
Flashlight: Enabling Innovation in Tools for Machine Learning
This is in part due to the difficulties involved in prototyping new computational paradigms with existing frameworks.
Rethinking Evaluation in ASR: Are Our Models Robust Enough?
Finally, we show that training a single acoustic model on the most widely-used datasets - combined - reaches competitive performance on both research and real-world benchmarks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Self-training and Pre-training are Complementary for Speech Recognition
Self-training and unsupervised pre-training have emerged as effective approaches to improve speech recognition systems using unlabeled data.
Ranked #1 on
Speech Recognition
on LibriSpeech train-clean-100 test-other
(using extra training data)
Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters
We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and over-all simplifying deployment of ASR systems that support diverse languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
DSCnet: Replicating Lidar Point Clouds with Deep Sensor Cloning
Recently, autonomous vehicles have created a demand for depth information, which is often obtained using hardware sensors such as Light detection and ranging (LIDAR).
