Search Results for author:
Found 57 papers, 35 papers with code
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning
Experimental results show that LHDFF outperforms existing audio captioning models.
E-PANNs: Sound Recognition Using Efficient Pre-trained Audio Neural Networks
Sounds carry an abundance of information about activities and events in our everyday environment, such as traffic noise, road works, music, or people talking.
Adapting Language-Audio Models as Few-Shot Audio Learners
We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data.
Compressing audio CNNs with graph centrality based filter pruning
For large-scale CNNs such as PANNs designed for audio tagging, our method reduces 24\% computations per inference with 41\% fewer parameters at a slight improvement in performance.
Efficient CNNs via Passive Filter Pruning
In comparison to the existing active filter pruning methods, the proposed pruning method is at least 4. 5 times faster in computing filter importance and is able to achieve similar performance compared to that of the active filter pruning methods.
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
To address this data scarcity issue, we introduce WavCaps, the first large-scale weakly-labelled audio captioning dataset, comprising approximately 400k audio clips with paired captions.
Leveraging Pre-trained AudioLDM for Text to Sound Generation: A Benchmark Study
Deep neural networks have recently achieved breakthroughs in sound generation with text prompts.
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
By learning the latent representations of audio signals and their compositions without modeling the cross-modal relationship, AudioLDM is advantageous in both generation quality and computational efficiency.
Ranked #5 on
Audio Generation
on AudioCaps
Towards Generating Diverse Audio Captions via Adversarial Training
Captions generated by existing models are generally faithful to the content of audio clips, however, these machine-generated captions are often deterministic (e. g., generating a fixed caption for a given audio clip), simple (e. g., using common words and simple grammar), and generic (e. g., generating the same caption for similar audio clips).
Ontology-aware Learning and Evaluation for Audio Tagging
The proposed metric, ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation.
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Audio captioning aims to generate text descriptions of audio clips.
Efficient Similarity-based Passive Filter Pruning for Compressing CNNs
However, the computational complexity of computing the pairwise similarity matrix is high, particularly when a convolutional layer has many filters.
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features
Moreover, in LHDFF, a new PANNs encoder is proposed called Residual PANNs (RPANNs) by fusing the low-dimensional feature from the intermediate convolution layer output and the high-dimensional feature from the final layer output of PANNs.
Learning the Spectrogram Temporal Resolution for Audio Classification
Starting from a high-temporal-resolution spectrogram such as one-millisecond hop size, we show that DiffRes can improve classification accuracy with the same computational complexity.
Simple Pooling Front-ends For Efficient Audio Classification
Recently, there has been increasing interest in building efficient audio neural networks for on-device scenarios.
Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains
Our system submitted to the DCASE 2022 Task 3 is based on our previous proposed Event-Independent Network V2 (EINV2) with a novel data augmentation method.
Low-complexity CNNs for Acoustic Scene Classification
This technical report describes the SurreyAudioTeam22s submission for DCASE 2022 ASC Task 1, Low-Complexity Acoustic Scene Classification (ASC).
Low-complexity CNNs for Acoustic Scene Classification
However, CNNs are resource hungry due to their large size and high computational complexity.
Segment-level Metric Learning for Few-shot Bioacoustic Event Detection
In addition, we use transductive inference on the validation set during training for better adaptation to novel classes.
Continual Learning For On-Device Environmental Sound Classification
Experimental results on the DCASE 2019 Task 1 and ESC-50 dataset show that our proposed method outperforms baseline continual learning methods on classification accuracy and computational efficiency, indicating our method can efficiently and incrementally learn new classes without the catastrophic forgetting problem for on-device environmental sound classification.
Automated Audio Captioning: An Overview of Recent Progress and New Challenges
In this paper, we present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets.
On Metric Learning for Audio-Text Cross-Modal Retrieval
We present an extensive evaluation of popular metric learning objectives on the AudioCaps and Clotho datasets.
A Passive Similarity based CNN Filter Pruning for Efficient Acoustic Scene Classification
We propose a passive filter pruning framework, where a few convolutional filters from the CNNs are eliminated to yield compressed CNNs.
Separate What You Describe: Language-Queried Audio Source Separation
In this paper, we introduce the task of language-queried audio source separation (LASS), which aims to separate a target source from an audio mixture based on a natural language query of the target source (e. g., "a man tells a joke followed by people laughing").
Deep Neural Decision Forest for Acoustic Scene Classification
In this paper, we propose a novel approach for ASC using deep neural decision forest (DNDF).
Leveraging Pre-trained BERT for Audio Captioning
BERT is a pre-trained language model that has been extensively used in Natural Language Processing (NLP) tasks.
Diverse Audio Captioning via Adversarial Training
As different people may describe an audio clip from different aspects using distinct words and grammars, we argue that an audio captioning system should have the ability to generate diverse captions for a fixed audio clip and across similar audio clips.
ARCA23K: An audio dataset for investigating open-set label noise
We show that the majority of labelling errors in ARCA23K are due to out-of-vocabulary audio clips, and we refer to this type of label noise as open-set label noise.
An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
Automated audio captioning aims to use natural language to describe the content of audio data.
Audio Captioning Transformer
In this paper, we propose an Audio Captioning Transformer (ACT), which is a full Transformer network based on an encoder-decoder architecture and is totally convolution-free.
Ranked #4 on
Audio captioning
on AudioCaps
CL4AC: A Contrastive Loss for Audio Captioning
Automated Audio captioning (AAC) is a cross-modal translation task that aims to use natural language to describe the content of an audio clip.
Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning
We evaluate our approach on the UrbanSound8K dataset, compared to SampleRNN, with the performance metrics measuring the quality and diversity of generated sounds.
Sound Event Detection: A Tutorial
The goal of automatic sound event detection (SED) methods is to recognize what is happening in an audio signal and when it is happening.
Gender Bias in Depression Detection Using Audio Features
Depression is a large-scale mental health problem and a challenging area for machine learning researchers in detection of depression.
An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection
Polyphonic sound event localization and detection (SELD), which jointly performs sound event detection (SED) and direction-of-arrival (DoA) estimation, detects the type and occurrence time of sound events as well as their corresponding DoA angles simultaneously.
Sound Audio and Speech Processing
Event-Independent Network for Polyphonic Sound Event Localization and Detection
In this paper, a novel event-independent network for polyphonic sound event localization and detection is proposed.
Audio and Speech Processing Sound
Learning with Out-of-Distribution Data for Audio Classification
The proposed method uses an auxiliary classifier, trained on data that is known to be in-distribution, for detection and relabelling.
Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation
In the spectrogram of a mixture of singing voices and music signals, there is usually more information about the voice in the low frequency bands than the high frequency bands.
Single-Channel Signal Separation and Deconvolution with Generative Adversarial Networks
Single-channel signal separation and deconvolution aims to separate and deconvolve individual sources from a single-channel mixture and is a challenging problem in which no prior knowledge of the mixing filters is available.
Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy
In this paper, it is experimentally shown that the training information of SED is able to contribute to the direction of arrival estimation (DOAE).
Sound Audio and Speech Processing
Referenceless Performance Evaluation of Audio Source Separation using Deep Neural Networks
Current performance evaluation for audio source separation depends on comparing the processed or separated signals with reference signals.
Sound Event Detection and Time-Frequency Segmentation from Weakly Labelled Data
Sound event detection (SED) aims to detect when and recognize what sound events happen in an audio clip.
Sound Audio and Speech Processing
Raw Multi-Channel Audio Source Separation using Multi-Resolution Convolutional Auto-Encoders
Supervised multi-channel audio source separation requires extracting useful spectral, temporal, and spatial features from the mixed signals.
A joint separation-classification model for sound event detection of weakly labelled data
First, we propose a separation mapping from the time-frequency (T-F) representation of an audio to the T-F segmentation masks of the audio events.
Sound Audio and Speech Processing
Audio Set classification with attention model: A probabilistic perspective
Then the classification of a bag is the expectation of the classification output of the instances in the bag with respect to the learned probability measure.
Sound Audio and Speech Processing
Multi-Resolution Fully Convolutional Neural Networks for Monaural Audio Source Separation
In deep neural networks with convolutional layers, each layer typically has fixed-size/single-resolution receptive field (RF).
Large-scale weakly supervised audio classification using gated convolutional neural network
In this paper, we present a gated convolutional neural network and a temporal attention-based localization method for audio classification, which won the 1st place in the large-scale weakly supervised sound event detection task of Detection and Classification of Acoustic Scenes and Events (DCASE) 2017 challenge.
Sound Audio and Speech Processing
Single Channel Audio Source Separation using Convolutional Denoising Autoencoders
Each CDAE is trained to separate one source and treats the other sources as background noise.
Sound 68T01 H.5.5; I.5; I.2.6; I.4.3
Attention and Localization based on a Deep Convolutional Recurrent Model for Weakly Supervised Audio Tagging
Audio tagging aims to perform multi-label classification on audio chunks and it is a newly proposed task in the Detection and Classification of Acoustic Scenes and Events 2016 (DCASE 2016) challenge.
Sound
Convolutional Gated Recurrent Neural Network Incorporating Spatial Features for Audio Tagging
In this paper, we propose to use a convolutional neural network (CNN) to extract robust features from mel-filter banks (MFBs), spectrograms or even raw waveforms for audio tagging.
Automatic Environmental Sound Recognition: Performance versus Computational Cost
In the context of the Internet of Things (IoT), sound sensing applications are required to run on embedded platforms where notions of product pricing and form factor impose hard constraints on the available computing power.
Unsupervised Feature Learning Based on Deep Models for Environmental Audio Tagging
For the unsupervised feature learning, we propose to use a symmetric or asymmetric deep de-noising auto-encoder (sDAE or aDAE) to generate new data-driven features from the Mel-Filter Banks (MFBs) features.
Hierarchical learning for DNN-based acoustic scene classification
In this paper, we present a deep neural network (DNN)-based acoustic scene classification framework.
Fully DNN-based Multi-label regression for audio tagging
Compared with the conventional Gaussian Mixture Model (GMM) and support vector machine (SVM) methods, the proposed fully DNN-based method could well utilize the long-term temporal information with the whole chunk as the input.
Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network
Identification and extraction of singing voice from within musical mixtures is a key challenge in source separation and machine audition.
Acoustic Scene Classification
We then describe a range of different algorithms submitted for a data challenge that was held to provide a general and fair benchmark for ASC techniques.
Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning
Feature learning can be performed at large scale and "unsupervised", meaning it requires no manual data labelling, yet it can improve performance on "supervised" tasks such as classification.
