Search Results for author:
Found 55 papers, 30 papers with code
Foundation Models for Music: A Survey
In recent years, foundation models (FMs) such as large language models (LLMs) and latent diffusion models (LDMs) have profoundly impacted diverse sectors, including music.
MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
Motivated by this, we introduce MuChoMusic, a benchmark for evaluating music understanding in multimodal language models focused on audio.
Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation
Yet scant research explores the details of how these LLMs perform on advanced music understanding and conditioned generation, especially from the multi-step reasoning perspective, which is a critical aspect in the conditioned, editable, and interactive human-computer co-creation process.
YourMT3+: Multi-instrument Music Transcription with Enhanced Transformer Architectures and Cross-dataset Stem Augmentation
Further testing on pop music recordings highlights the limitations of current models.
Ranked #1 on
Multi-instrument Music Transcription
on URMP
(using extra training data)
Towards Building an End-to-End Multilingual Automatic Lyrics Transcription Model
Furthermore, we demonstrate that incorporating language information significantly enhances performance.
Automatic Lyrics Transcription
Automatic Speech Recognition
+1
MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series
To improve the transparency of LLMs, the research community has formed to open-source truly open LLMs (e. g., Pythia, Amber, OLMo), where more details (e. g., pre-training corpus and training code) are being provided.
Explaining models relating objects and privacy
To explain the decision of these models, we use feature-attribution to identify and quantify which objects (and which of their features) are more relevant to privacy classification with respect to a reference input (i. e., no objects localised in an image) predicted as public.
ComposerX: Multi-Agent Symbolic Music Composition with LLMs
Music composition represents the creative side of humanity, and itself is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints.
MuPT: A Generative Symbolic Music Pretrained Transformer
In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music.
Mind the Domain Gap: a Systematic Analysis on Bioacoustic Sound Event Detection
A recent development in the field is the introduction of the task known as few-shot bioacoustic sound event detection, which aims to train a versatile animal sound detector using only a small set of audio samples.
Generalized Multi-Source Inference for Text Conditioned Music Diffusion Models
Multi-Source Diffusion Models (MSDM) allow for compositional musical generation tasks: generating a set of coherent sources, creating accompaniments, and performing source separation.
WavCraft: Audio Editing and Generation with Large Language Models
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing.
ChatMusician: Understanding and Generating Music Intrinsically with LLM
It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language.
A Data-Driven Analysis of Robust Automatic Piano Transcription
Algorithms for automatic piano transcription have improved dramatically in recent years due to new datasets and modeling techniques.
Ranked #3 on
Music Transcription
on MAPS
(using extra training data)
Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities
Moreover, we improve the framework of audio language model by using interleaved audio-text embeddings as the input sequence.
The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation
We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models.
ATGNN: Audio Tagging Graph Neural Network
Deep learning models such as CNNs and Transformers have achieved impressive performance for end-to-end audio tagging.
MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
Recognizing the significance of pitch in capturing the nuances of IPTs and the importance of onset in locating IPT events, we investigate multi-task finetuning with pitch and onset detection as auxiliary tasks.
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response
Large Language Models (LLMs) have shown immense potential in multimodal applications, yet the convergence of textual and musical domains remains not well-explored.
From West to East: Who can understand the music of the others better?
This leads to research questions on whether these models can be used to learn representations for different music cultures and styles, or whether we can build similar music audio embedding models trained on data from different cultures or styles.
On the Effectiveness of Speech Self-supervised Learning for Music
Our findings suggest that training with music data can generally improve performance on MIR tasks, even when models are trained using paradigms designed for speech.
LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT
We introduce LyricWhiz, a robust, multilingual, and zero-shot automatic lyrics transcription method achieving state-of-the-art performance on various lyrics transcription datasets, even in challenging genres such as rock and metal.
MARBLE: Music Audio Representation Benchmark for Universal Evaluation
This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark.
MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training
Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored.
Few-shot Class-incremental Audio Classification Using Dynamically Expanded Classifier with Self-attention Modified Prototypes
Labeled support samples and unlabeled query samples are used to train the prototype adaptation network and update the classifier, since they are informative for audio classification.
Adapting Language-Audio Models as Few-Shot Audio Learners
We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data.
MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning
The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL).
Learning Music Representations with wav2vec 2.0
In addition, the results are superior to the pre-trained model on speech embeddings, demonstrating that wav2vec 2. 0 pre-trained on music data can be a promising music representation model.
Contrastive Audio-Language Learning for Music
In this work, we explore cross-modal learning in an attempt to bridge audio and language in the music domain.
Anomalous behaviour in loss-gradient based interpretability methods
Loss-gradients are used to interpret the decision making process of deep learning models.
Deep Conditional Representation Learning for Drum Sample Retrieval by Vocalisation
Imitating musical instruments with the human voice is an efficient way of communicating ideas between music producers, from sketching melody lines to clarifying desired sonorities.
Exploring Transformer's potential on automatic piano transcription
Most recent research about automatic music transcription (AMT) uses convolutional neural networks and recurrent neural networks to model the mapping from music signals to symbolic notation.
A Comparison of Deep Learning MOS Predictors for Speech Synthesis Quality
In this paper, we evaluate several MOS predictors based on wav2vec 2. 0 and the NISQA speech quality prediction model to explore the role of the training data, the influence of the system type, and the role of cross-domain features in SSL models.
Learning music audio representations via weak language supervision
To address this question, we design a multimodal architecture for music and language pre-training (MuLaP) optimised via a set of proxy tasks.
An evaluation of data augmentation methods for sound scene geotagging
Sound scene geotagging is a new topic of research which has evolved from acoustic scene classification.
Joint Scattering for Automatic Chick Call Recognition
Animal vocalisations contain important information about health, emotional state, and behaviour, thus can be potentially used for animal welfare monitoring.
More for Less: Non-Intrusive Speech Quality Assessment with Limited Annotations
This paper indicates that multi-task learning combined with feature representations from unlabelled data is a promising approach to deal with the lack of large MOS annotated datasets.
Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes
We also include ablation studies investigating the effects of the use of multiple kernel shapes and comparing different input representations for the audio and the note-related information.
MusCaps: Generating Captions for Music Audio
Content-based music information retrieval has seen rapid progress with the adoption of deep learning.
Adversarial Unsupervised Domain Adaptation for Harmonic-Percussive Source Separation
This paper addresses the problem of domain adaptation for the task of music source separation.
The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracy
We attempt to use only the pitch labels (together with spectrogram reconstruction loss) and explore how far this model can go without introducing supervised sub-tasks.
Reliable Local Explanations for Machine Listening
One way to analyse the behaviour of machine learning models is through local explanations that highlight input features that maximally influence model predictions.
Memory Controlled Sequential Self Attention for Sound Recognition
In this paper we investigate the importance of the extent of memory in sequential self attention for sound recognition.
Musical Features for Automatic Music Transcription Evaluation
This technical report gives a detailed, formal description of the features introduced in the paper: Adrien Ycart, Lele Liu, Emmanouil Benetos and Marcus T. Pearce.
Audio Impairment Recognition Using a Correlation-Based Feature Representation
Audio impairment recognition is based on finding noise in audio files and categorising the impairment type.
Modeling plate and spring reverberation using a DSP-informed deep neural network
Plate and spring reverberators are electromechanical systems first used and researched as means to substitute real room reverberation.
Adversarial Attacks in Sound Event Classification
In addition, we show that the adversarial attacks are very effective across the different models.
A general-purpose deep learning approach to model time-varying audio effects
Audio processors whose parameters are modified periodically over time are often referred as time-varying or modulation based audio effects.
GAN-based Generation and Automatic Selection of Explanations for Neural Networks
However, this requires a careful selection of hyper-parameters to generate interpretable examples for each neuron of interest, and current methods rely on a manual, qualitative evaluation of each setting, which is prohibitively slow.
Ensemble Models for Spoofing Detection in Automatic Speaker Verification
Our ensemble model outperforms all our single models and the baselines from the challenge for both attack types.
Audio and Speech Processing Sound
Optimal Neural Network Feature Selection for Spatial-Temporal Forecasting
In this paper, we show empirical evidence on how to construct the optimal feature selection or input representation used by the input layer of a feedforward neural network for the propose of forecasting spatial-temporal signals.
Sound Event Detection in Synthetic Audio: Analysis of the DCASE 2016 Task Results
As part of the 2016 public evaluation challenge on Detection and Classification of Acoustic Scenes and Events (DCASE 2016), the second task focused on evaluating sound event detection systems using synthetic mixtures of office sounds.
An End-to-End Neural Network for Polyphonic Piano Music Transcription
We compare performance of the neural network based acoustic models with two popular unsupervised acoustic models.
An evaluation framework for event detection using a morphological model of acoustic scenes
This paper introduces a model of environmental acoustic scenes which adopts a morphological approach by ab-stracting temporal structures of acoustic scenes.
A Hybrid Recurrent Neural Network For Music Transcription
We investigate the problem of incorporating higher-level symbolic score-like information into Automatic Music Transcription (AMT) systems to improve their performance.
