Search Results for author:
Found 16 papers, 10 papers with code
An Empirical Study of Speech Language Models for Prompt-Conditioned Speech Synthesis
Speech language models (LMs) are promising for high-quality speech synthesis through in-context learning.
MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation
There have been emerging research interest and advances in speech-to-speech translation (S2ST), translating utterances from one language to another.
Seamless: Multilingual Expressive and Streaming Speech Translation
In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion.
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
Ranked #1 on
Machine Translation
on flores95-devtest eng-X
Automatic Speech Recognition
Speech-to-Speech Translation
+3
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
We enhance the model performance by subword prediction in the first-pass decoder, advanced two-pass decoder architecture design and search strategy, and better training regularization.
Improving Speech-to-Speech Translation Through Unlabeled Text
Direct speech-to-speech translation (S2ST) is among the most challenging problems in the translation paradigm due to the significant scarcity of S2ST data.
Named Entity Detection and Injection for Direct Speech Translation
In a sentence, certain words are critical for its semantic.
Simple and Effective Unsupervised Speech Translation
The amount of labeled data to train models for speech tasks is limited for most languages, however, the data scarcity is exacerbated for speech translation which requires labeled data covering two different languages.
Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models
In many natural language processing (NLP) tasks the same input (e. g. source sentence) can have multiple possible outputs (e. g. translations).
Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling
From these observations, we conclude that the high degree of oversmoothing is the main reason behind the degenerate case of overly probable short sequences in a neural autoregressive model.
Mode recovery in neural autoregressive sequence modeling
We propose to study these phenomena by investigating how the modes, or local maxima, of a distribution are maintained throughout the full learning chain of the ground-truth, empirical, learned and decoding-induced distributions, via the newly proposed mode recovery cost.
Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition.
Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training
Generative dialogue models currently suffer from a number of problems which standard maximum likelihood training does not address.
Neural Text Generation with Unlikelihood Training
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core.
Multi-Turn Beam Search for Neural Dialogue Modeling
We propose a novel approach for conversation-level inference by explicitly modeling the dialogue partner and running beam search across multiple conversation turns.
Importance of Search and Evaluation Strategies in Neural Dialogue Modeling
We investigate the impact of search strategies in neural dialogue modeling.
