Search Results for author:
Found 18 papers, 11 papers with code
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning
Experimental results show that LHDFF outperforms existing audio captioning models.
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
To address this data scarcity issue, we introduce WavCaps, the first large-scale weakly-labelled audio captioning dataset, comprising approximately 400k audio clips with paired captions.
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
By learning the latent representations of audio signals and their compositions without modeling the cross-modal relationship, AudioLDM is advantageous in both generation quality and computational efficiency.
Ranked #3 on
Audio Generation
on AudioCaps
Towards Generating Diverse Audio Captions via Adversarial Training
Captions generated by existing models are generally faithful to the content of audio clips, however, these machine-generated captions are often deterministic (e. g., generating a fixed caption for a given audio clip), simple (e. g., using common words and simple grammar), and generic (e. g., generating the same caption for similar audio clips).
Ontology-aware Learning and Evaluation for Audio Tagging
The proposed metric, ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation.
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Audio captioning aims to generate text descriptions of audio clips.
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features
Moreover, in LHDFF, a new PANNs encoder is proposed called Residual PANNs (RPANNs) by fusing the low-dimensional feature from the intermediate convolution layer output and the high-dimensional feature from the final layer output of PANNs.
Simple Pooling Front-ends For Efficient Audio Classification
Recently, there has been increasing interest in building efficient audio neural networks for on-device scenarios.
Segment-level Metric Learning for Few-shot Bioacoustic Event Detection
In addition, we use transductive inference on the validation set during training for better adaptation to novel classes.
Automated Audio Captioning: An Overview of Recent Progress and New Challenges
In this paper, we present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets.
On Metric Learning for Audio-Text Cross-Modal Retrieval
We present an extensive evaluation of popular metric learning objectives on the AudioCaps and Clotho datasets.
Separate What You Describe: Language-Queried Audio Source Separation
In this paper, we introduce the task of language-queried audio source separation (LASS), which aims to separate a target source from an audio mixture based on a natural language query of the target source (e. g., "a man tells a joke followed by people laughing").
Deep Neural Decision Forest for Acoustic Scene Classification
In this paper, we propose a novel approach for ASC using deep neural decision forest (DNDF).
Leveraging Pre-trained BERT for Audio Captioning
BERT is a pre-trained language model that has been extensively used in Natural Language Processing (NLP) tasks.
Diverse Audio Captioning via Adversarial Training
As different people may describe an audio clip from different aspects using distinct words and grammars, we argue that an audio captioning system should have the ability to generate diverse captions for a fixed audio clip and across similar audio clips.
An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
Automated audio captioning aims to use natural language to describe the content of audio data.
Audio Captioning Transformer
In this paper, we propose an Audio Captioning Transformer (ACT), which is a full Transformer network based on an encoder-decoder architecture and is totally convolution-free.
Ranked #4 on
Audio captioning
on AudioCaps
CL4AC: A Contrastive Loss for Audio Captioning
Automated Audio captioning (AAC) is a cross-modal translation task that aims to use natural language to describe the content of an audio clip.
