Audio Generation
49 papers with code • 3 benchmarks • 7 datasets
Audio generation (synthesis) is the task of generating raw audio such as speech.
( Image credit: MelNet )
Benchmarks
These leaderboards are used to track progress in Audio Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
AudioCaps
NSynth
MAESTRO
MovieNet
NES-MDB
ADL Piano MIDI
Most implemented papers
WaveNet: A Generative Model for Raw Audio
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms.
Adversarial Audio Synthesis
chrisdonahue/wavegan
•
•
ICLR 2019
Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales.
GANSynth: Adversarial Neural Audio Synthesis
tensorflow/magenta
•
•
ICLR 2019
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence.
MelNet: A Generative Model for Audio in the Frequency Domain
fatchord/MelNet
•
•
Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps.
It's Raw! Audio Generation with State-Space Models
SaShiMi yields state-of-the-art performance for unconditional waveform generation in the autoregressive setting.
SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
soroushmehr/sampleRNN_ICLR2017
•
•
In this paper we propose a novel model for unconditional audio generation based on generating one audio sample at a time.
Audio Super Resolution using Neural Networks
We introduce a new audio processing technique that increases the sampling rate of signals such as speech or music using deep convolutional neural networks.
AudioLM: a Language Modeling Approach to Audio Generation
suno-ai/bark
•
•
We introduce AudioLM, a framework for high-quality audio generation with long-term consistency.
Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion
liusongxiang/StarGAN-Voice-Conversion
•
•
NeurIPS 2019
End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations.
DDSP: Differentiable Digital Signal Processing
In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods.
