Search Results for author:
Found 40 papers, 20 papers with code
On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization
The emergence of various notions of ``consistency'' in diffusion models has garnered considerable attention and helped achieve improved sample quality, likelihood estimation, and accelerated sampling.
Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders
Diffusion-based speech enhancement (SE) has been investigated recently, but its decoding is very time-consuming.
The Whole Is Greater than the Sum of Its Parts: Improving DNN-based Music Source Separation
We modify the target network, i. e., the network architecture of the original DNN-based MSS, by adding bridging paths for each output instrument to share their information.
Diffusion-based Signal Refiner for Speech Separation
We experimentally show that our refiner can provide a clearer harmonic structure of speech and improves the reference-free metric of perceptual quality for arbitrary preceding model architectures.
PeaCoK: Persona Commonsense Knowledge for Consistent and Engaging Narratives
They must also learn to maintain consistent speaker personas for themselves throughout the narrative, so that their counterparts feel involved in a realistic conversation or story.
Cross-modal Face- and Voice-style Transfer
Image-to-image translation and voice conversion enable the generation of a new facial image and voice while maintaining some of the semantics such as a pose in an image and linguistic content in audio, respectively.
Adversarially Slicing Generative Networks: Discriminator Slices Feature for One-Dimensional Optimal Transport
Generative adversarial networks (GANs) learn a target probability distribution by optimizing a generator and a discriminator with minimax objectives.
GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration
Pre-trained diffusion models have been successfully used as priors in a variety of linear inverse problems, where the goal is to reconstruct a signal from noisy linear measurements.
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Further, videos in the wild often contain off-screen sounds and background noise that may hinder the model from learning the desired audio-textual correspondence.
Unsupervised vocal dereverberation with diffusion-based generative models
Removing reverb from reverberant music is a necessary technique to clean up audio for downstream music manipulations.
Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects
We propose an end-to-end music mixing style transfer system that converts the mixing style of an input multitrack to that of a reference song.
Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
Although deep neural network (DNN)-based speech enhancement (SE) methods outperform the previous non-DNN-based ones, they often degrade the perceptual quality of generated outputs.
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge
Understanding rich narratives, such as dialogues and stories, often requires natural language processing systems to access relevant knowledge from commonsense knowledge graphs.
Robust One-Shot Singing Voice Conversion
In this work, we propose robust one-shot SVC (ROSVC) that performs any-to-any SVC robustly even on such distorted singing voices using less than 10s of a reference voice.
Hierarchical Diffusion Models for Singing Voice Neural Vocoder
Recent progress in deep generative models has improved the quality of neural vocoders in speech domain.
DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT).
Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation
Score-based generative models learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise.
Automatic music mixing with deep learning and out-of-domain data
Music mixing traditionally involves recording instruments in the form of clean, individual tracks and blending them into a final mixture using audio effects and expert knowledge (e. g., a mixing engineer).
STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events
Additionally, the report presents the baseline system that accompanies the dataset in the challenge with emphasis on the differences with the baseline of the previous iterations; namely, introduction of the multi-ACCDOA representation to handle multiple simultaneous occurences of events of the same class, and support for additional improved input features for the microphone array format.
Ranked #1 on
Sound Event Localization and Detection
on STARSS22
SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization
In this paper, we propose a new training scheme that extends the standard VAE via novel stochastic dequantization and quantization, called stochastically quantized variational autoencoder (SQ-VAE).
Distortion Audio Effects: Learning How to Recover the Clean Signal
Given the recent advances in music source separation and automatic mixing, removing audio effects in music tracks is a meaningful step toward developing an automated remixing system.
Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training
The multi- ACCDOA format (a class- and track-wise output format) enables the model to solve the cases with overlaps from the same class.
Automatic DJ Transitions with Differentiable Audio Effects and Generative Adversarial Networks
A central task of a Disc Jockey (DJ) is to create a mixset of mu-sic with seamless transitions between adjacent tracks.
Spatial mixup: Directional loudness modification as data augmentation for sound event localization and detection
Data augmentation methods have shown great importance in diverse supervised learning problems where labeled data is scarce or costly to obtain.
Music Demixing Challenge 2021
The main differences compared with the past challenges are 1) the competition is designed to more easily allow machine learning practitioners from other disciplines to participate, 2) evaluation is done on a hidden test set created by music professionals dedicated exclusively to the challenge to assure the transparency of the challenge, i. e., the test set is not accessible from anyone except the challenge organizers, and 3) the dataset provides a wider range of music genres and involved a greater number of mixing engineers.
Ensemble of ACCDOA- and EINV2-based Systems with D3Nets and Impulse Response Simulation for Sound Event Localization and Detection
This report describes our systems submitted to the DCASE2021 challenge task 3: sound event localization and detection (SELD) with directional interference.
Densely Connected Multi-Dilated Convolutional Networks for Dense Prediction Tasks
In this paper, we claim the importance of a dense simultaneous modeling of multiresolution representation and propose a novel CNN architecture called densely connected multidilated DenseNet (D3Net).
Training Speech Enhancement Systems with Noisy Speech Datasets
Furthermore, we propose a noise augmentation scheme for mixture-invariant training (MixIT), which allows using it also in such scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Preventing Oversmoothing in VAE via Generalized Variance Parameterization
Variational autoencoders (VAEs) often suffer from posterior collapse, which is a phenomenon in which the learned latent space becomes uninformative.
Hierarchical disentangled representation learning for singing voice conversion
Conventional singing voice conversion (SVC) methods often suffer from operating in high-resolution audio owing to a high dimensionality of data.
AR-ELBO: Preventing Posterior Collapse Induced by Oversmoothing in Gaussian VAE
Variational autoencoders (VAEs) often suffer from posterior collapse, which is a phenomenon that the learned latent space becomes uninformative.
Densely connected multidilated convolutional networks for dense prediction tasks
In this paper, we claim the importance of a dense simultaneous modeling of multiresolution representation and propose a novel CNN architecture called densely connected multidilated DenseNet (D3Net).
Ranked #43 on
Semantic Segmentation
on Cityscapes test
ACCDOA: Activity-Coupled Cartesian Direction of Arrival Representation for Sound Event Localization and Detection
Conventional NN-based methods use two branches for a sound event detection (SED) target and a direction-of-arrival (DOA) target.
All for One and One for All: Improving Music Separation by Bridging Networks
This paper proposes several improvements for music separation with deep neural networks (DNNs), namely a multi-domain loss (MDL) and two combination schemes.
Ranked #19 on
Music Source Separation
on MUSDB18
Adversarial attacks on audio source separation
Despite the excellent performance of neural-network-based audio source separation methods and their wide range of applications, their robustness against intentional attacks has been largely neglected.
D3Net: Densely connected multidilated DenseNet for music source separation
In this paper, we claim the importance of a rapid growth of a receptive field and a simultaneous modeling of multi-resolution data in a single convolution layer, and propose a novel CNN architecture called densely connected dilated DenseNet (D3Net).
Ranked #11 on
Music Source Separation
on MUSDB18
(using extra training data)
Improving Voice Separation by Incorporating End-to-end Speech Recognition
Despite recent advances in voice separation methods, many challenges remain in realistic scenarios such as noisy recording and the limits of available data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Improving DNN-based Music Source Separation using Phase Features
Music source separation with deep neural networks typically relies only on amplitude features.
MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation
Deep neural networks have become an indispensable technique for audio source separation (ASS).
Ranked #15 on
Music Source Separation
on MUSDB18
(using extra training data)
Music Source Separation
Sound
Audio and Speech Processing
Multi-scale Multi-band DenseNets for Audio Source Separation
This paper deals with the problem of audio source separation.
