Speech Synthesis
110 papers with code • 2 benchmarks • 12 datasets
Speech synthesis is the task of generating speech from some other modality like text, lip movements etc.
Please note that the leaderboards here are not really comparable between studies - as they use mean opinion score as a metric and collect different samples from Amazon Mechnical Turk.
( Image credit: WaveNet: A generative model for raw audio )
Benchmarks
| Trend | Dataset | Best Model | Paper Title | Paper | Code | Compare |
|---|---|---|---|---|---|---|
|
||||||
|
Datasets
THCHS-30
TaL Corpus
Greatest papers with code
Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
CorentinJ/Real-Time-Voice-Cloning
•
•
NeurIPS 2018
Clone a voice in 5 seconds to generate arbitrary speech in real-time
Efficient Neural Audio Synthesis
CorentinJ/Real-Time-Voice-Cloning
•
•
ICML 2018
The small number of weights in a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile CPU in real time.
Tacotron: Towards End-to-End Speech Synthesis
CorentinJ/Real-Time-Voice-Cloning
•
•
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.
Ranked #4 on
Speech Synthesis
on North American English
A Spectral Energy Distance for Parallel Speech Synthesis
google-research/google-research
•
•
NeurIPS 2020
Speech synthesis is an important practical generative modeling problem that has seen great progress over the last few years, with likelihood-based autoregressive neural models now outperforming traditional concatenative systems.
WaveNet: A Generative Model for Raw Audio
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
Ranked #1 on
Speech Synthesis
on Mandarin Chinese
Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion
espnet/espnet
•
•
With these data, three neural TTS models -- Tacotron2, Transformer and FastSpeech are applied for building bilingual and code-switched TTS.
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
NVIDIA/tacotron2
•
•
This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text.
Ranked #1 on
Speech Synthesis
on North American English
Handling Background Noise in Neural Speech Generation
Recent advances in neural-network based generative modeling of speech has shown great potential for speech coding.
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
TensorSpeech/TensorflowTTS
•
•
NeurIPS 2020
Several recent work on speech synthesis have employed generative adversarial networks (GANs) to produce raw waveforms.
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
TensorSpeech/TensorflowTTS
•
•
ICLR 2021
In this paper, we propose FastSpeech 2, which addresses the issues in FastSpeech and better solves the one-to-many mapping problem in TTS by 1) directly training the model with ground-truth target instead of the simplified output from teacher, and 2) introducing more variation information of speech (e. g., pitch, energy and more accurate duration) as conditional inputs.