Search Results for author:
Found 51 papers, 13 papers with code
PIE: A Parallel Idiomatic Expression Corpus for Idiomatic Sentence Generation and Paraphrasing
Idiomatic expressions (IE) play an important role in natural language, and have long been a “pain in the neck” for NLP systems.
Rich Syntactic and Semantic Information Helps Unsupervised Text Style Transfer
Text style transfer aims to change an input sentence to an output sentence by changing its text style while preserving the content.
Findings of the IWSLT 2022 Evaluation Campaign
The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation.
AV-Flow: Transforming Text to Audio-Visual Human-like Interactions
Through extensive experiments, we show that our method outperforms prior work, synthesizing natural-looking 4D talking avatars.
DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models
Spoken language models (SLMs) have gained increasing attention with advancements in text-based, decoder-only language models.
Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents
Despite broad interest in modeling spoken dialogue agents, most approaches are inherently "half-duplex" -- restricted to turn-based interaction with responses requiring explicit prompting by the user or implicit tracking of interruption or silence events.
Investigating Decoder-only Large Language Models for Speech-to-text Translation
Large language models (LLMs), known for their exceptional reasoning capabilities, generalizability, and fluency across diverse domains, present a promising avenue for enhancing speech-related tasks.
Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation
In this paper, we propose a textless acoustic model with a self-supervised distillation strategy for noise-robust expressive speech-to-speech translation (S2ST).
SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thought
Expressive speech-to-speech translation (S2ST) is a key research topic in seamless communication, which focuses on the preservation of semantics and speaker vocal style in translated speech.
MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation
There have been emerging research interest and advances in speech-to-speech translation (S2ST), translating utterances from one language to another.
An Empirical Study of Speech Language Models for Prompt-Conditioned Speech Synthesis
Speech language models (LMs) are promising for high-quality speech synthesis through in-context learning.
Seamless: Multilingual Expressive and Streaming Speech Translation
In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion.
automatic-speech-translation
Multimodal Machine Translation
+2
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
Ranked #1 on
Speech-to-Speech Translation
on CVSS
(using extra training data)
Automatic Speech Recognition
Speech-to-Speech Translation
+4
Multilingual Speech-to-Speech Translation into Multiple Target Languages
Despite a few studies on multilingual S2ST, their focus is the multilinguality on the source side, i. e., the translation from multiple source languages to one target language.
Pre-training for Speech Translation: CTC Meets Optimal Transport
Nevertheless, CTC is only a partial solution and thus, in our second contribution, we propose a novel pre-training method combining CTC and optimal transport to further reduce this gap.
A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translation
Expressive speech-to-speech translation (S2ST) aims to transfer prosodic attributes of source speech to target speech while maintaining translation accuracy.
Speech-to-Speech Translation For A Real-world Unwritten Language
We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings.
Improving Speech-to-Speech Translation Through Unlabeled Text
Direct speech-to-speech translation (S2ST) is among the most challenging problems in the translation paradigm due to the significant scarcity of S2ST data.
Named Entity Detection and Injection for Direct Speech Translation
In a sentence, certain words are critical for its semantic.
T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation
We present a new approach to perform zero-shot cross-modal transfer between speech and text for translation tasks.
Unified Speech-Text Pre-training for Speech Translation and Recognition
Two pre-training configurations for speech translation and recognition, respectively, are presented to alleviate subtask interference.
Efficient Language Modeling with Sparse all-MLP
All-MLP architectures have attracted increasing interest as an alternative to attention-based models.
Ranked #17 on
Question Answering
on StoryCloze
Idiomatic Expression Paraphrasing without Strong Supervision
In this paper, we study the task of idiomatic sentence paraphrasing (ISP), which aims to paraphrase a sentence with an IE by replacing the IE with its literal paraphrase.
Textless Speech-to-Speech Translation on Real Data
To our knowledge, we are the first to establish a textless S2ST technique that can be trained with real-world data and works for multiple language pairs.
Multimodal and Multilingual Embeddings for Large-Scale Speech Mining
Using a similarity metric in that multimodal embedding space, we perform mining of audio in German, French, Spanish and English from Librivox against billions of sentences from Common Crawl.
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention
We present a direct simultaneous speech-to-speech translation (Simul-S2ST) model, Furthermore, the generation of translation is independent from intermediate text representations.
Simultaneous Speech-to-Speech Translation
Speech Synthesis
+2
From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translation
Speech-to-speech translation (S2ST) converts input speech to speech in another language.
Data Augmentation
Simultaneous Speech-to-Speech Translation
+4
Contrastive Clustering to Mine Pseudo Parallel Data for Unsupervised Translation
Modern unsupervised machine translation systems mostly train their models by generating synthetic parallel training data from large unlabeled monolingual corpora of different languages through various means, such as iterative back-translation.
FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task
In this paper, we describe our end-to-end multilingual speech translation system submitted to the IWSLT 2021 evaluation campaign on the Multilingual Speech Translation shared task.
Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling
We further propose attention sharing strategies to facilitate parameter sharing and specialization in multilingual and multi-domain sequence modeling.
LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models
Cross-lingual document representations enable language understanding in multilingual contexts and allow transfer learning from high-resource to low-resource languages at the document level.
Abusive Language Detection in Heterogeneous Contexts: Dataset Collection and the Role of Supervised Attention
Abusive language is a massive problem in online social platforms.
Adaptive Sparse Transformer for Multilingual Translation
Based on these insights, we propose an adaptive and sparse architecture for multilingual modeling, and train the model to learn shared and language-specific parameters to improve the positive transfer and mitigate the interference.
Robust Optimization for Multilingual Translation with Imbalanced Data
We show that common training method which upsamples low resources can not robustly optimize population loss with risks of either underfitting high resource languages or overfitting low resource ones.
From Solving a Problem Boldly to Cutting the Gordian Knot: Idiomatic Text Generation
We study a new application for text generation -- idiomatic sentence generation -- which aims to transfer literal phrases in sentences into their idiomatic counterparts.
Self-Supervised Euphemism Detection and Identification for Content Moderation
It is usually apparent to a human moderator that a word is being used euphemistically, but they may not know what the secret meaning is, and therefore whether the message violates policy.
Enriching Word Embeddings with Temporal and Spatial Information
The meaning of a word is closely linked to sociocultural factors that can change over time and location, resulting in corresponding meaning changes.
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Metaphors are rhetorical use of words based on the conceptual mapping as opposed to their literal use.
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
In this paper, we study machine reading comprehension (MRC) on long texts, where a model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer.
FUSE: Multi-Faceted Set Expansion by Coherent Clustering of Skip-grams
In this paper, we study the task of multi-faceted set expansion, which aims to capture all semantic facets in the seed set and return multiple sets of entities, one for each semantic facet.
PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space
Finding the right reviewers to assess the quality of conference submissions is a time consuming process for conference organizers.
Equipping Educational Applications with Domain Knowledge
One of the challenges of building natural language processing (NLP) applications for education is finding a large domain-specific corpus for the subject of interest (e. g., history or science).
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia
We present an approach based on multilingual sentence embeddings to automatically extract parallel sentences from the content of Wikipedia articles in 85 languages, including several dialects or low-resource languages.
Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus
Our generator employs an attention-based encoder-decoder to transfer a sentence from the source style to the target style.
Document Similarity for Texts of Varying Lengths via Hidden Topics
This is because of the lexical, contextual and the abstraction gaps between a long document of rich details and its concise summary of abstract information.
Context-Sensitive Malicious Spelling Error Correction
Misspelled words of the malicious kind work by changing specific keywords and are intended to thwart existing automated applications for cyber-environment control such as harassing content detection on the Internet and email spam detection.
Preposition Sense Disambiguation and Representation
Prepositions are highly polysemous, and their variegated senses encode significant semantic information.
Embedding Syntax and Semantics of Prepositions via Tensor Decomposition
Prepositions are among the most frequent words in English and play complex roles in the syntax and semantics of sentences.
Prepositions in Context
Prepositions are highly polysemous, and their variegated senses encode significant semantic information.
Geometry of Compositionality
This paper proposes a simple test for compositionality (i. e., literal usage) of a word or phrase in a context-specific way.
