Search Results for author:
Found 29 papers, 15 papers with code
Long-form music generation with latent diffusion
Audio-based generative models for music have seen great strides recently, but so far have not managed to produce full-length music tracks with coherent musical structure.
Fast Timing-Conditioned Latent Audio Diffusion
Generating long-form 44. 1kHz stereo audio from text prompts can be computationally demanding.
GASS: Generalizing Audio Source Separation with Large-scale Data
Here, we study a single general audio source separation (GASS) model trained to separate speech, music, and sound events in a supervised fashion with a large-scale dataset.
Mono-to-stereo through parametric stereo generation
Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific panning of sound elements.
CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Our results show the effectiveness of the proposed method, and that the pretrained diffusion prior can reduce the modality transfer gap.
Towards Robust Image-in-Audio Deep Steganography
The field of steganography has experienced a surge of interest due to the recent advancements in AI-powered techniques, particularly in the context of multimodal setups that enable the concealment of signals within signals of a different nature.
Ranked #1 on
Image Reconstruction
on Audio Set
Full-band General Audio Synthesis with Score-based Diffusion
Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental sounds.
Adversarial Permutation Invariant Training for Universal Sound Separation
Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models that do so.
Universal Speech Enhancement with Score-based Diffusion
We hope that both our methodology and technical contributions encourage researchers and practitioners to adopt a universal approach to speech enhancement, possibly framing it as a generative task.
On loss functions and evaluation metrics for music source separation
We investigate which loss functions provide better separations via benchmarking an extensive set of those for music source separation.
Upsampling layers for music source separation
Upsampling artifacts are caused by problematic upsampling layers and due to spectral replicas that emerge while upsampling.
On tuning consistent annealed sampling for denoising score matching
Score-based generative models provide state-of-the-art quality for image and audio synthesis.
Multichannel-based learning for audio object extraction
The current paradigm for creating and deploying immersive audio content is based on audio objects, which are composed of an audio track and position metadata.
Sound Audio and Speech Processing
On permutation invariant training for speech source separation
We study permutation invariant training (PIT), which targets at the permutation ambiguity problem for speaker independent source separation models.
Upsampling artifacts in neural audio synthesis
We then compare different upsampling layers, showing that nearest neighbor upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts.
Automatic multitrack mixing with a differentiable mixing console of neural audio effects
Applications of deep learning to automatic multitrack mixing are largely unexplored.
Audio and Speech Processing Sound
SESQA: semi-supervised learning for speech quality assessment
Automatic speech quality assessment is an important, transversal task whose progress is hampered by the scarcity of human annotations, poor generalization to unseen recording conditions, and a lack of flexibility of existing approaches.
FSD50K: An Open Dataset of Human-Labeled Sound Events
Most existing datasets for sound event recognition (SER) are relatively small and/or domain-specific, with the exception of AudioSet, based on over 2M tracks from YouTube videos and encompassing over 500 sound classes.
TensorFlow Audio Models in Essentia
Essentia is a reference open-source C++/Python library for audio and music analysis.
An empirical study of Conv-TasNet
Furthermore, we offer insights into the generalization capabilities of Conv-TasNet and the potential value of improvements to the encoder/decoder.
musicnn: Pre-trained convolutional neural networks for music audio tagging
Pronounced as "musician", the musicnn library contains a set of pre-trained musically motivated convolutional neural networks for music audio tagging: https://github. com/jordipons/musicnn.
End-to-end music source separation: is it possible in the waveform domain?
Most of the currently successful source separation techniques use the magnitude spectrogram as input, and are therefore by default omitting part of the signal: the phase.
Ranked #26 on
Music Source Separation
on MUSDB18
Training neural audio classifiers with few data
We investigate supervised learning strategies that improve the training of neural network audio classifiers on small annotated collections.
General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline
The goal of the task is to build an audio tagging system that can recognize the category of an audio clip from a subset of 41 diverse categories drawn from the AudioSet Ontology.
Randomly weighted CNNs for (music) audio classification
The computer vision literature shows that randomly weighted neural networks perform reasonably as feature extractors.
Sound Audio and Speech Processing
End-to-end learning for music audio tagging at scale
The lack of data tends to limit the outcomes of deep learning research, particularly when dealing with end-to-end learning stacks processing raw data such as waveforms.
Sound Audio and Speech Processing
Audio to score matching by combining phonetic and duration information
We approach the singing phrase audio to score matching problem by using phonetic and duration information - with a focus on studying the jingju a cappella singing case.
Sound
A Wavenet for Speech Denoising
In order to overcome this limitation, we propose an end-to-end learning method for speech denoising based on Wavenet.
Sound
Timbre Analysis of Music Audio Signals with Convolutional Neural Networks
The focus of this work is to study how to efficiently tailor Convolutional Neural Networks (CNNs) towards learning timbre representations from log-mel magnitude spectrograms.
Sound
