Search Results for author:
Found 21 papers, 11 papers with code
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
We propose a cross-lingual neural codec language model, VALL-E X, for cross-lingual speech synthesis.
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
In addition, we find Vall-E could preserve the speaker's emotion and acoustic environment of the acoustic prompt in synthesis.
BEATs: Audio Pre-Training with Acoustic Tokenizers
In the first iteration, we use random projection as the acoustic tokenizer to train an audio SSL model in a mask and label prediction manner.
Ranked #1 on
Audio Classification
on Balanced Audio Set
Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
Recently, masked prediction pre-training has seen remarkable progress in self-supervised learning (SSL) for speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?
Recently, self-supervised learning (SSL) has demonstrated strong performance in speaker recognition, even if the pre-training objective is designed for speech recognition.
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Recently, pioneer work finds that speech pre-trained models can solve full-stack speech processing tasks, because the model utilizes bottom layers to learn speaker-related information and top layers to encode content-related information.
Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
The reconstruction module is used for auxiliary learning to improve the noise robustness of the learned representation and thus is not required during inference.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+8
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks.
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
We integrate the proposed methods into the HuBERT framework.
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
In this paper we propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech via contrastive learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
UniSpeech at scale: An Empirical Study of Pre-training Method on Large-Scale Speech Recognition Dataset
Recently, there has been a vast interest in self-supervised learning (SSL) where the model is pre-trained on large scale unlabeled data and then fine-tuned on a small labeled dataset.
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC learning and phonetically-aware contrastive self-supervised learning are conducted in a multi-task learning manner.
Continuous Speech Separation with Conformer
Continuous speech separation plays a vital role in complicated speech related tasks such as conversation transcription.
Ranked #1 on
Speech Separation
on LibriCSS
(using extra training data)
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Among all three E2E models, transformer-AED achieved the best accuracy in both streaming and non-streaming mode.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Curriculum Pre-training for End-to-End Speech Translation
End-to-end speech translation poses a heavy burden on the encoder, because it has to transcribe, understand, and learn cross-lingual semantics simultaneously.
Low Latency End-to-End Streaming Speech Recognition with a Scout Network
The attention-based Transformer model has achieved promising results for speech recognition (SR) in the offline mode.
Audio and Speech Processing
Semantic Mask for Transformer based End-to-End Speech Recognition
Attention-based encoder-decoder model has achieved impressive results for both automatic speech recognition (ASR) and text-to-speech (TTS) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation
End-to-end speech translation, a hot topic in recent years, aims to translate a segment of audio into a specific language with an end-to-end model.
Source Dependency-Aware Transformer with Supervised Self-Attention
Recently, Transformer has achieved the state-of-the-art performance on many machine translation tasks.
Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network
Due to the highly parallelizable architecture, Transformer is faster to train than RNN-based models and popularly used in machine translation tasks.
