Search Results for author:
Found 28 papers, 13 papers with code
HARP 2.0: Expanding Hosted, Asynchronous, Remote Processing for Deep Learning in the DAW
HARP 2. 0 brings deep learning models to digital audio workstation (DAW) software through hosted, asynchronous, remote processing, allowing users to route audio from a plug-in interface through any compatible Gradio endpoint to perform arbitrary transformations.
Code Drift: Towards Idempotent Neural Audio Codecs
Neural codecs have demonstrated strong performance in high-fidelity compression of audio signals at low bitrates.
Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects
This work introduces Text2FX, a method that leverages CLAP embeddings and differentiable digital signal processing to control audio effects, such as equalization and reverberation, using open-vocabulary natural language prompts (e. g., "make this sound in-your-face and bold").
Fine-Grained and Interpretable Neural Speech Editing
Fine-grained editing of speech attributes$\unicode{x2014}$such as prosody (i. e., the pitch, loudness, and phoneme durations), pronunciation, speaker identity, and formants$\unicode{x2014}$is useful for fine-tuning and fixing imperfections in human and AI-generated speech recordings for creation of podcasts, film dialogue, and video game dialogue.
High-Fidelity Neural Phonetic Posteriorgrams
A phonetic posteriorgram (PPG) is a time-varying categorical distribution over acoustic units of speech (e. g., phonemes).
Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Model
Every artist has a creative process that draws inspiration from previous artists and their works.
Crowdsourced and Automatic Speech Prominence Estimation
Speech prominence estimation is the process of assigning a numeric value to the prominence of each word in an utterance.
VampNet: Music Generation via Masked Acoustic Token Modeling
We introduce VampNet, a masked acoustic token modeling approach to music synthesis, compression, inpainting, and variation.
Cross-domain Neural Pitch and Periodicity Estimation
We also introduce a novel entropy-based method for extracting periodicity and per-frame voiced-unvoiced classifications from statistical inference-based pitch estimators (e. g., neural networks), and show how to train a neural pitch estimator to simultaneously handle both speech and music data (i. e., cross-domain estimation) without performance degradation.
Music Separation Enhancement with Generative Modeling
Despite phenomenal progress in recent years, state-of-the-art music separation systems produce source estimates with significant perceptual shortcomings, such as adding extraneous noise or removing harmonics.
Reproducible Subjective Evaluation
ReSEval lets researchers launch A/B, ABX, Mean Opinion Score (MOS) and MUltiple Stimuli with Hidden Reference and Anchor (MUSHRA) tests on audio, image, text, or video data from a command-line interface or using one line of Python, making it as easy to run as objective evaluation.
Deep Learning Tools for Audacity: Helping Researchers Expand the Artist's Toolkit
We present a software framework that integrates neural networks into the popular open-source audio editing software, Audacity, with a minimal amount of developer effort.
Unsupervised Source Separation By Steering Pretrained Music Models
We showcase an unsupervised method that repurposes deep models trained for music generation and music tagging for audio source separation, without any retraining.
Neural Pitch-Shifting and Time-Stretching with Controllable LPCNet
Modifying the pitch and timing of an audio signal are fundamental audio editing operations with applications in speech manipulation, audio-visual synchronization, and singing voice editing and synthesis.
Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition
Deep learning work on musical instrument recognition has generally focused on instrument classes for which we have abundant data.
Context-Aware Prosody Correction for Text-Based Speech Editing
Text-based speech editors expedite the process of editing speech recordings by permitting editing via intuitive cut, copy, and paste operations on a speech transcript.
A Study of Transfer Learning in Music Source Separation
Supervised deep learning methods for performing audio source separation can be very effective in domains where there is a large amount of training data.
Bespoke Neural Networks for Score-Informed Source Separation
In this paper, we introduce a simple method that can separate arbitrary musical instruments from an audio mixture.
AutoClip: Adaptive Gradient Clipping for Source Separation Networks
Clipping the gradient is a known approach to improving gradient descent, but requires hand selection of a clipping threshold hyperparameter.
Bach or Mock? A Grading Function for Chorales in the Style of J.S. Bach
Deep generative systems that learn probabilistic models from a corpus of existing music do not explicitly encode knowledge of a musical style, compared to traditional rule-based systems.
Incorporating Music Knowledge in Continual Dataset Augmentation for Music Generation
In this paper, we present augmentative generation (Aug-Gen), a method of dataset augmentation for any music generation system trained on a resource-constrained domain.
OtoMechanic: Auditory Automobile Diagnostics via Query-by-Example
Many automobile components in need of repair produce characteristic sounds.
Bootstrapping deep music separation from primitive auditory grouping principles
They are trained on synthetic mixtures of audio made from isolated sound source recordings so that ground truth for the separation is known.
Model selection for deep audio source separation via clustering analysis
We compare our confidence-based ensemble approach to using individual models with no selection, to an oracle that always selects the best model and to a random model selector.
Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments
We present a single deep learning architecture that can both separate an audio recording of a musical mixture into constituent single-instrument recordings and transcribe these instruments into a human-readable format at the same time, learning a shared musical representation for both tasks.
Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures
These estimates, together with a weighting scheme in the time-frequency domain, based on confidence in the separation quality, are used to train a deep learning model that can be used for single-channel separation, where no source direction information is available.
VocalSet: A Singing Voice Dataset
We present VocalSet, a singing voice dataset of a capella singing.
An Overview of Lead and Accompaniment Separation in Music
For model-based methods, we organize them according to whether they concentrate on the lead signal, the accompaniment, or both.
Sound Audio and Speech Processing
