Search Results for author:
Found 28 papers, 24 papers with code
WavCraft: Audio Editing and Generation with Natural Language Prompts
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing.
Description on IEEE ICME 2024 Grand Challenge: Semi-supervised Acoustic Scene Classification under Domain Shift
In addition, considering the abundance of unlabeled acoustic scene data in the real world, it is important to study the possible ways to utilize these unlabelled data.
Retrieval-Augmented Text-to-Audio Generation
Despite recent progress in text-to-audio (TTA) generation, we show that the state-of-the-art models, such as AudioLDM, trained on datasets with an imbalanced class distribution, such as AudioCaps, are biased in their generation performance.
Ranked #2 on
Audio Generation
on AudioCaps
AudioSR: Versatile Audio Super-resolution at Scale
Audio super-resolution is a fundamental task that predicts high-frequency components for low-resolution audio, enhancing audio quality in digital applications.
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Any audio can be translated into LOA based on AudioMAE, a self-supervised pre-trained representation learning model.
Ranked #3 on
Audio Generation
on AudioCaps
Separate Anything You Describe
In this work, we introduce AudioSep, a foundation model for open-domain audio source separation with natural language queries.
MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
Diffusion models have shown promising results in cross-modal generation tasks, including text-to-image and text-to-audio generation.
WavJourney: Compositional Audio Creation with Large Language Models
Subjective evaluations demonstrate the potential of WavJourney in crafting engaging storytelling audio content from text.
Text-Driven Foley Sound Generation With Latent Diffusion Model
We have observed that the feature embedding extracted by the text encoder can significantly affect the performance of the generation model.
E-PANNs: Sound Recognition Using Efficient Pre-trained Audio Neural Networks
Sounds carry an abundance of information about activities and events in our everyday environment, such as traffic noise, road works, music, or people talking.
Adapting Language-Audio Models as Few-Shot Audio Learners
We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data.
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
To address this data scarcity issue, we introduce WavCaps, the first large-scale weakly-labelled audio captioning dataset, comprising approximately 400k audio clips with paired captions.
Ranked #1 on
Zero-Shot Environment Sound Classification
on ESC-50
(using extra training data)
Leveraging Pre-trained AudioLDM for Text to Sound Generation: A Benchmark Study
Deep neural networks have recently achieved breakthroughs in sound generation with text prompts.
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
By learning the latent representations of audio signals and their compositions without modeling the cross-modal relationship, AudioLDM is advantageous in both generation quality and computational efficiency.
Ranked #9 on
Audio Generation
on AudioCaps
ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech
Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples.
Ontology-aware Learning and Evaluation for Audio Tagging
The proposed metric, ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation.
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Audio captioning aims to generate text descriptions of audio clips.
Learning Temporal Resolution in Spectrogram for Audio Classification
The audio spectrogram is a time-frequency representation that has been widely used for audio classification.
Simple Pooling Front-ends For Efficient Audio Classification
Recently, there has been increasing interest in building efficient audio neural networks for on-device scenarios.
Segment-level Metric Learning for Few-shot Bioacoustic Event Detection
In addition, we use transductive inference on the validation set during training for better adaptation to novel classes.
BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis
Combining this novel perspective of two-stage synthesis with advanced generative models (i. e., the diffusion models), the proposed BinauralGrad is able to generate accurate and high-fidelity binaural audio samples.
NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
In this paper, we answer these questions by first defining the human-level quality based on the statistical significance of subjective measure and introducing appropriate guidelines to judge it, and then developing a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset.
Ranked #1 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration
Speech restoration aims to remove distortions in speech signals.
Separate What You Describe: Language-Queried Audio Source Separation
In this paper, we introduce the task of language-queried audio source separation (LASS), which aims to separate a target source from an audio mixture based on a natural language query of the target source (e. g., "a man tells a joke followed by people laughing").
Neural Vocoder is All You Need for Speech Super-resolution
In this paper, we propose a neural vocoder based speech super-resolution method (NVSR) that can handle a variety of input resolution and upsampling ratios.
Ranked #2 on
Audio Super-Resolution
on VCTK Multi-Speaker
Leveraging Pre-trained BERT for Audio Captioning
BERT is a pre-trained language model that has been extensively used in Natural Language Processing (NLP) tasks.
CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNet
On the MUSDB18HQ test set, we propose a 276-layer CWS-PResUNet and achieve state-of-the-art (SoTA) performance on vocals with an 8. 92 signal-to-distortion ratio (SDR) score.
Ranked #11 on
Music Source Separation
on MUSDB18-HQ
Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Music
We aim to address the major issues in CNN-based high-resolution MSS model: high computational cost and weight sharing between distinctly different bands.
Audio and Speech Processing Sound
