Search Results for author:
Found 23 papers, 9 papers with code
Video-Guided Foley Sound Generation with Multimodal Controls
MultiFoley also allows users to choose reference audio from sound effects (SFX) libraries or partial videos for conditioning.
Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learning
Audio-text contrastive models have become a powerful approach in music representation learning.
Bridging High-Quality Audio and Video via Language for Sound Effects Retrieval from Visual Queries
We show that our system, trained using our automatic data curation pipeline, significantly outperforms baselines trained on in-the-wild data on the task of HQ SFX retrieval for video.
Language-Guided Music Recommendation for Video via Prompt Analogies
A key challenge of this problem setting is that existing music video datasets provide the needed (video, music) training pairs, but lack text descriptions of the music.
Efficient Spoken Language Recognition via Multilabel Classification
Spoken language recognition (SLR) is the task of automatically identifying the language present in a speech signal.
Conditional Generation of Audio from Video via Foley Analogies
Second, we propose a model for generating a soundtrack for a silent input video, given a user-supplied example that specifies what the video should "sound like".
Language-Guided Audio-Visual Source Separation via Trimodal Consistency
We propose a self-supervised approach for learning to perform audio source separation in videos based on natural language queries, using only unlabeled video and audio pairs as training data.
It's Time for Artistic Correspondence in Music and Video
In order to capture the high-level concepts that are required to solve the task, we propose modeling the long-term temporal context of both the video and the music signals, using Transformer networks for each modality.
Filler Word Detection and Classification: A Dataset and Benchmark
In this work, we present a novel speech dataset, PodcastFillers, with 35K annotated filler words and 50K annotations of other sounds that commonly occur in podcasts such as breaths, laughter, and word repetitions.
HEAR: Holistic Evaluation of Audio Representations
The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios.
Emotion Embedding Spaces for Matching Music to Stories
Content creators often use music to enhance their stories, as it can be a powerful tool to convey emotion.
Soundata: A Python library for reproducible use of audio datasets
Soundata is a Python library for loading and working with audio datasets in a standardized way, removing the need for writing custom loaders in every project, and improving reproducibility by providing tools to validate data against a canonical version.
What's All the FUSS About Free Universal Sound Separation Data?
We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound types.
SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context
In this article, we describe our data collection procedure and propose evaluation metrics for multilabel classification of urban sound tags.
Disentangled Multidimensional Metric Learning for Music Similarity
For this task, it is typically necessary to define a similarity metric to compare one recording to another.
Metric Learning vs Classification for Disentangled Music Representation Learning
For this, we (1) outline past work on the relationship between metric learning and classification, (2) extend this relationship to multi-label data by exploring three different learning approaches and their disentangled versions, and (3) evaluate all models on four tasks (training time, similarity retrieval, auto-tagging, and triplet prediction).
Controllable Neural Prosody Synthesis
Speech synthesis has recently seen significant improvements in fidelity, driven by the advent of neural vocoders and neural prosody generators.
Telling Left from Right: Learning Spatial Correspondence of Sight and Sound
Self-supervised audio-visual learning aims to capture useful representations of video by leveraging correspondences between visual and audio inputs.
Sound event detection in domestic environments withweakly labeled data and soundscape synthesis
This paper presents Task 4 of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2019 challenge and provides a first analysis of the challenge results.
Ranked #9 on
Sound Event Detection
on DESED
Robust sound event detection in bioacoustic sensor networks
As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise.
Adaptive pooling operators for weakly labeled sound event detection
In this work, we treat SED as a multiple instance learning (MIL) problem, where training labels are static over a short excerpt, indicating the presence or absence of sound sources but not their temporal locality.
CREPE: A Convolutional Representation for Pitch Estimation
To date, the best performing techniques, such as the pYIN algorithm, are based on a combination of DSP pipelines and heuristics.
Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a "shallow" dictionary learning model with augmentation.
