Search Results for author:
Found 30 papers, 11 papers with code
Isochrony-Controlled Speech-to-Text Translation: A study on translating from Sino-Tibetan to Indo-European Languages
End-to-end speech translation (ST), which translates source language speech directly into target language text, has garnered significant attention in recent years.
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
This paper introduces VALL-E 2, the latest advancement in neural codec language models that marks a milestone in zero-shot text-to-speech synthesis (TTS), achieving human parity for the first time.
TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation
There is a rising interest and trend in research towards directly translating speech from one language to another, known as end-to-end speech-to-speech translation.
CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations
In this paper, we introduce CoVoMix: Conversational Voice Mixture Generation, a novel model for zero-shot, human-like, multi-speaker, multi-round dialogue speech generation.
DDTSE: Discriminative Diffusion Model for Target Speech Extraction
In this paper, we introduce the Discriminative Diffusion model for Target Speech Extraction (DDTSE).
Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech Recognition
Most end-to-end (E2E) speech recognition models are composed of encoder and decoder blocks that perform acoustic and language modeling functions.
Adapting Multi-Lingual ASR Models for Handling Multiple Talkers
State-of-the-art large-scale universal speech models (USMs) show a decent automatic speech recognition (ASR) performance across multiple domains and languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation
Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language.
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities.
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models which lack generative abilities.
Code-Switching Text Generation and Injection in Mandarin-English ASR
Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Target Sound Extraction with Variable Cross-modality Clues
Automatic target sound extraction (TSE) is a machine learning approach to mimic the human auditory perception capability of attending to a sound source of interest from a mixture of sources.
Deploying self-supervised learning in the wild for hybrid automatic speech recognition
Self-supervised learning (SSL) methods have proven to be very successful in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data
In this way, the decoder learns to reconstruct original speech information with codes before learning to generate correct text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Recently, pioneer work finds that speech pre-trained models can solve full-stack speech processing tasks, because the model utilizes bottom layers to learn speaker-related information and top layers to encode content-related information.
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
The reconstruction module is used for auxiliary learning to improve the noise robustness of the learned representation and thus is not required during inference.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+9
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks.
Ranked #1 on
Speech Recognition
on CALLHOME En
Multilingual Speech Recognition using Knowledge Transfer across Learning Processes
Multilingual end-to-end(E2E) models have shown a great potential in the expansion of the language coverage in the realm of automatic speech recognition(ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+8
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
We integrate the proposed methods into the HuBERT framework.
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
In this paper we propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech via contrastive learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
UniSpeech at scale: An Empirical Study of Pre-training Method on Large-Scale Speech Recognition Dataset
Recently, there has been a vast interest in self-supervised learning (SSL) where the model is pre-trained on large scale unlabeled data and then fine-tuned on a small labeled dataset.
Speech-language Pre-training for End-to-end Spoken Language Understanding
End-to-end (E2E) spoken language understanding (SLU) can infer semantics directly from speech signal without cascading an automatic speech recognizer (ASR) with a natural language understanding (NLU) module.
Ranked #3 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC learning and phonetically-aware contrastive self-supervised learning are conducted in a multi-task learning manner.
To Trust, or Not to Trust? A Study of Human Bias in Automated Video Interview Assessments
Supervised systems require human labels for training.
A Report on the 2017 Native Language Identification Shared Task
We believe this makes for a more interesting shared task while building on the methods and results from the previous two shared tasks.
A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has been shown to be very effective for modeling and predicting sequential data, e. g. speech utterances or handwritten documents.
Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network
Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has been shown to be very effective for tagging sequential data, e. g. speech utterances or handwritten documents.
