Search Results for author:
Found 66 papers, 20 papers with code
From Aesthetics to Human Preferences: Comparative Perspectives of Evaluating Text-to-Music Systems
In this paper, we use music generation as a case study to investigate the gap between automatic evaluation metrics and human preferences.
Heterogeneous bimodal attention fusion for speech emotion recognition
To address this problem, we propose a novel framework called Heterogeneous Bimodal Attention Fusion (HBAF) for multi-level multi-modal interaction in conversational emotion recognition.
Bimodal Connection Attention Fusion for Speech Emotion Recognition
In this work, we propose Bimodal Connection Attention Fusion (BCAF) method, which includes three main modules: the interactive connection network, the bimodal attention network, and the correlative attention network.
LHGNN: Local-Higher Order Graph Neural Networks For Audio Classification and Tagging
Transformers have set new benchmarks in audio processing tasks, leveraging self-attention mechanisms to capture complex patterns and dependencies within audio data.
Ranked #1 on
Audio Classification
on FSD50K
(Mean AP metric)
Single-word Auditory Attention Decoding Using Deep Learning Model
This approach is largely unexplored in the field of AAD but leads to a single-word auditory attention decoding problem in which an epoch of an EEG signal timed to a specific word is labeled as attended or unattended.
MixNet: Joining Force of Classical and Modern Approaches Toward the Comprehensive Pipeline in Motor Imagery EEG Classification
Recent advances in deep learning (DL) have significantly impacted motor imagery (MI)-based brain-computer interface (BCI) systems, enhancing the decoding of electroencephalography (EEG) signals.
Data-Centric Human Preference Optimization with Rationales
Reinforcement learning from human feedback plays a crucial role in aligning language models towards human preferences, traditionally represented through comparisons between pairs or sets of responses within a given context.
1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem
Previous work has utilised class weighted loss for training, but problems remain as it sometimes causes over-fitting for minor classes or under-fitting for major classes.
Fish-bone diagram of research issue: Gain a bird's-eye view on a specific research topic
Novice researchers often face difficulties in understanding a multitude of academic papers and grasping the fundamentals of a new research field.
Mind the Domain Gap: a Systematic Analysis on Bioacoustic Sound Event Detection
A recent development in the field is the introduction of the task known as few-shot bioacoustic sound event detection, which aims to train a versatile animal sound detector using only a small set of audio samples.
WavCraft: Audio Editing and Generation with Large Language Models
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing.
Hierarchical Tree-structured Knowledge Graph For Academic Insight Survey
To address these issues, this study aims to support research insight surveys for beginner researchers by establishing a hierarchical tree-structured knowledge graph that reflects the inheritance insight of research topics and the relevance insight among the academic papers.
DisDet: Exploring Detectability of Backdoor Attack on Diffusion Models
In this paper, for the first time, we systematically explore the detectability of the poisoned noise input for the backdoored diffusion models, an important performance metric yet little explored in the existing works.
ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks
Low-rank compression, a popular model compression technique that produces compact convolutional neural networks (CNNs) with low rankness, has been well-studied in the literature.
Improving the Robustness of 3D Human Pose Estimation: A Benchmark and Learning from Noisy Input
We observe the poor generalization of state-of-the-art 3D pose lifters in the presence of corruption and establish two techniques to tackle this issue.
Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities
In this work, we introduce Acoustic Prompt Tuning (APT), a new adapter extending LLMs and VLMs to the audio domain by injecting audio embeddings to the input of LLMs, namely soft prompting.
ATGNN: Audio Tagging Graph Neural Network
Deep learning models such as CNNs and Transformers have achieved impressive performance for end-to-end audio tagging.
Adapting Language-Audio Models as Few-Shot Audio Learners
We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data.
CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities
The results indicate that training the model on multimodal data does positively influence performance when tested on unimodal data.
Ranked #2 on
Sleep Stage Detection
on SHHS
An Inception-Residual-Based Architecture with Multi-Objective Loss for Detecting Respiratory Anomalies
Notably, we achieved the Top-1 performance in Task 2-1 and Task 2-2 with the highest Score of 74. 5% and 53. 9%, respectively.
cross-modal fusion techniques for utterance-level emotion recognition from text and speech
Multimodal emotion recognition (MER) is a fundamental complex research problem due to the uncertainty of human emotional expression and the heterogeneity gap between different modalities.
deep learning of segment-level feature representation for speech emotion recognition in conversations
Accurately detecting emotions in conversation is a necessary yet challenging task due to the complexity of emotions and dynamics in dialogues.
L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging
In this work, we show that while encoding the logic of a whole sleep cycle is crucial to improve sleep staging performance, the sequential modelling approach in existing state-of-the-art deep learning models are inefficient for that purpose.
Improving trajectory localization accuracy via direction-of-arrival derivative estimation
In this paper, we do a comprehensive analysis of improvement in sound source localization by combining the direction of arrivals (DOAs) with their derivatives which quantify the changes in the positions of sources over time.
CSTAR: Towards Compact and STructured Deep Neural Networks with Adversarial Robustness
Considering the co-importance of model compactness and robustness in practical applications, several prior works have explored to improve the adversarial robustness of the sparse neural networks.
Modelling black-box audio effects with time-varying feature modulation
Deep learning approaches for black-box modelling of audio effects have shown promise, however, the majority of existing work focuses on nonlinear effects with behaviour on relatively short time-scales, such as guitar amplifiers and distortion.
Personalized Longitudinal Assessment of Multiple Sclerosis Using Smartphones
The results show that the proposed model is promising to achieve personalized longitudinal MS assessment; they also suggest that features related to gait and balance as well as upper extremity function, remotely collected from sensor-based assessments, may be useful digital markers for predicting MS over time.
RIBAC: Towards Robust and Imperceptible Backdoor Attack against Compact DNN
Recently backdoor attack has become an emerging threat to the security of deep neural network (DNN) models.
Polyphonic audio event detection: multi-label or multi-class multi-task classification problem?
The network is composed of a backbone subnet and multiple task-specific subnets.
Feature matching as improved transfer learning technique for wearable EEG
This method consists of training a model with larger amounts of data from the source modality and few paired samples of source and target modality.
SALSA-Lite: A Fast and Effective Feature for Polyphonic Sound Event Localization and Detection with Microphone Arrays
In this work, we introduce SALSA-Lite, a fast and effective feature for polyphonic SELD using microphone array inputs.
Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions
Modern deep learning holds a great potential to transform clinical practice on human sleep.
CHIP: CHannel Independence-based Pruning for Compact Neural Networks
Filter pruning has been widely used for neural network compression because of its enabled practical acceleration.
Neural Synthesis of Footsteps Sound Effects with Generative Adversarial Networks
Footsteps are among the most ubiquitous sound effects in multimedia applications.
Pediatric Automatic Sleep Staging: A comparative study of state-of-the-art deep learning methods
Background: Despite the tremendous progress recently made towards automatic sleep staging in adults, it is currently unknown if the most advanced algorithms generalize to the pediatric population, which displays distinctive characteristics in overnight polysomnography (PSG).
SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification
It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level.
Light-weight sleep monitoring: electrode distance matters more than placement for automatic scoring
Modern sleep monitoring development is shifting towards the use of unobtrusive sensors combined with algorithms for automatic sleep scoring.
Multi-view Audio and Music Classification
The learned embedding in the subnetworks are then concatenated to form the multi-view embedding for classification similar to a simple concatenation network.
MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification
We integrate deep metric learning into a multi-task autoencoder to learn a compact and discriminative latent representation from EEG and perform classification simultaneously.
Inception-Based Network and Multi-Spectrogram Ensemble Applied For Predicting Respiratory Anomalies and Lung Diseases
This paper presents an inception-based deep neural network for detecting lung diseases using respiratory sound input.
Self-Attention Generative Adversarial Network for Speech Enhancement
Existing generative adversarial networks (GANs) for speech enhancement solely rely on the convolution operation, which may obscure temporal dependencies across the sequence input.
On Multitask Loss Function for Audio Event Detection and Localization
Audio event localization and detection (SELD) have been commonly tackled using multitask models.
XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging
This work proposes a sequence-to-sequence sleep staging model, XSleepNet, that is capable of learning a joint representation from both raw signals and time-frequency images.
Ranked #1 on
Sleep Stage Detection
on PhysioNet Challenge 2018
Personalized Automatic Sleep Staging with Single-Night Data: a Pilot Study with KL-Divergence Regularization
We employ the pretrained SeqSleepNet (i. e. the subject independent model) as a starting point and finetune it with the single-night personalization data to derive the personalized model.
MetaSleepLearner: A Pilot Study on Fast Adaptation of Bio-signals-Based Sleep Stage Classifier to New Individual Subject Using Meta-Learning
This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording.
CNN-MoE based framework for classification of respiratory anomalies and lung disease detection
This paper presents and explores a robust deep learning framework for auscultation analysis.
Robust Deep Learning Framework For Predicting Respiratory Anomalies and Diseases
This paper presents a robust deep learning framework developed to detect respiratory diseases from recordings of respiratory sounds.
Improving GANs for Speech Enhancement
The former constrains the generators to learn a common mapping that is iteratively applied at all enhancement stages and results in a small model footprint.
CAG: A Real-time Low-cost Enhanced-robustness High-transferability Content-aware Adversarial Attack Generator
In addition, CAG exhibits high transferability across different DNN classifier models in black-box attack scenario by introducing random dropout in the process of generating perturbations.
Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning
We employ the Montreal Archive of Sleep Studies (MASS) database consisting of 200 subjects as the source domain and study deep transfer learning on three different target domains: the Sleep Cassette subset and the Sleep Telemetry subset of the Sleep-EDF Expanded database, and the Surrey-cEEGrid database.
Ranked #1 on
Multimodal Sleep Stage Detection
on Surrey-PSG
Automatic Sleep Stage Classification
Multimodal Sleep Stage Detection
+2
Deep Transfer Learning for Single-Channel Automatic Sleep Staging with Channel Mismatch
This work presents a deep transfer learning approach to overcome the channel mismatch problem and transfer knowledge from a large dataset to a small cohort to study automatic sleep staging with single-channel input.
Spatio-Temporal Attention Pooling for Audio Scene Classification
Acoustic scenes are rich and redundant in their content.
Beyond Equal-Length Snippets: How Long is Sufficient to Recognize an Audio Scene?
Moreover, as model fusion with deep network ensemble is prevalent in audio scene classification, we further study whether, and if so, when model fusion is necessary for this task.
Unifying Isolated and Overlapping Audio Event Detection with Multi-Label Multi-Task Convolutional Recurrent Neural Networks
We propose a multi-label multi-task framework based on a convolutional recurrent neural network to unify detection of isolated and overlapping audio events.
SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging
At the sequence processing level, a recurrent layer placed on top of the learned epoch-wise features for long-term modelling of sequential epochs.
Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification
While the proposed framework is orthogonal to the widely adopted classification schemes, which take one or multiple epochs as contextual inputs and produce a single classification decision on the target epoch, we demonstrate its advantages in several ways.
Ranked #2 on
Sleep Stage Detection
on MASS SS2
Enabling Early Audio Event Detection with Neural Networks
The proposed system consists of a novel inference step coupled with dual parallel tailored-loss deep neural networks (DNNs).
DNN and CNN with Weighted and Multi-task Loss Functions for Audio Event Detection
Our proposed systems significantly outperform the challenge baseline, improving F-score from 72. 7% to 90. 0% and reducing detection error rate from 0. 53 to 0. 18 on average on the development data.
Audio Scene Classification with Deep Recurrent Neural Networks
We introduce in this work an efficient approach for audio scene classification using deep recurrent neural networks.
Classifying Variable-Length Audio Files with All-Convolutional Networks and Masked Global Pooling
We trained a deep all-convolutional neural network with masked global pooling to perform single-label classification for acoustic scene classification and multi-label classification for domestic audio tagging in the DCASE-2016 contest.
CaR-FOREST: Joint Classification-Regression Decision Forests for Overlapping Audio Event Detection
The regression phase is then carried out to let the positive audio segments vote for the event onsets and offsets, and therefore model the temporal structure of audio events.
CNN-LTE: a Class of 1-X Pooling Convolutional Neural Networks on Label Tree Embeddings for Audio Scene Recognition
This category taxonomy is then used in the feature extraction step in which an audio scene instance is represented by a label tree embedding image.
Label Tree Embeddings for Acoustic Scene Classification
We present in this paper an efficient approach for acoustic scene classification by exploring the structure of class labels.
Learning Compact Structural Representations for Audio Events Using Regressor Banks
The entries of the descriptor are produced by evaluating a set of regressors on the input signal.
Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks
We present in this paper a simple, yet efficient convolutional neural network (CNN) architecture for robust audio event recognition.
Comparing Time and Frequency Domain for Audio Event Recognition Using Deep Learning
Recognizing acoustic events is an intricate problem for a machine and an emerging field of research.
