Search Results for author:
Found 105 papers, 52 papers with code
Estimating Musical Surprisal in Audio
In modeling musical surprisal expectancy with computational methods, it has been proposed to use the information content (IC) of one-step predictions from an autoregressive model as a proxy for surprisal in symbolic music.
Effective Pre-Training of Audio Transformers for Sound Event Detection
For five transformers, we obtain a substantial performance improvement over previously available checkpoints both on AudioSet frame-level predictions and on frame-level sound event detection downstream tasks, confirming our pipeline's effectiveness.
Improving Query-by-Vocal Imitation with Contrastive Learning and Audio Pretraining
Query-by-Vocal Imitation (QBV) is about searching audio files within databases using vocal imitations created by the user's voice.
Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval
We evaluate our method on the ClothoV2 and the AudioCaps benchmark and show that it improves retrieval performance, even in a restricting self-distillation setting where a single model generates and then learns from the estimated correspondences.
Ranked #1 on
Text to Audio Retrieval
on Clotho
(using extra training data)
Controlling Surprisal in Music Generation via Information Content Curve Matching
This enables the comparison of surprisal across different musical content even if the musical events occur in irregular time intervals.
Quantifying the Corpus Bias Problem in Automatic Music Transcription Systems
Automatic Music Transcription (AMT) is the task of recognizing notes in audio recordings of music.
TheGlueNote: Learned Representations for Robust and Flexible Note Alignment
Note alignment refers to the task of matching individual notes of two versions of the same symbolically encoded piece.
Improving Audio Spectrogram Transformers for Sound Event Detection Through Multi-Stage Training
We fine-tune three large Audio Spectrogram Transformers, PaSST, BEATs, and ATST, on the joint DESED and MAESTRO datasets in a two-stage training procedure.
GraphMuse: A Library for Symbolic Music Graph Processing
Addressing this gap, we present GraphMuse, a graph processing framework and library that facilitates efficient music graph processing and GNN training for symbolic music tasks.
Multi-Iteration Multi-Stage Fine-Tuning of Transformers for Sound Event Detection with Heterogeneous Datasets
A single model and an ensemble, both based on our proposed training procedure, ranked first in Task 4 of the DCASE Challenge 2024.
Cluster and Separate: a GNN Approach to Voice and Staff Prediction for Score Engraving
To aid the qualitative analysis of our results, we support the export in symbolic music formats and provide a direct visualization of our outputs graph over the musical score.
Fusing Audio and Metadata Embeddings Improves Language-based Audio Retrieval
Matching raw audio signals with textual descriptions requires understanding the audio's content and the description's semantics and then drawing connections between the two modalities.
DExter: Learning and Controlling Performance Expression with Diffusion Models
The perceptual-feature-conditioned generation and transferring capabilities of DExter are verified by a proxy model predicting perceptual characteristics of differently steered performances.
Data-Efficient Low-Complexity Acoustic Scene Classification in the DCASE 2024 Challenge
This article describes the Data-Efficient Low-Complexity Acoustic Scene Classification Task in the DCASE 2024 Challenge and the corresponding baseline system.
Perception-Inspired Graph Convolution for Music Understanding Tasks
We propose a new graph convolutional block, called MusGConv, specifically designed for the efficient processing of musical score data and motivated by general perceptual principles.
Expressivity-aware Music Performance Retrieval using Mid-level Perceptual Features and Emotion Word Embeddings
On the text side, we use emotion-enriched word embeddings (EWE) and on the audio side, we extract mid-level perceptual features instead of generic audio embeddings.
Are we describing the same sound? An analysis of word embedding spaces of expressive piano performance
Using a music research dataset of free text performance characterizations and a follow-up study sorting the annotations into clusters, we derive a ground truth for a domain-specific semantic similarity structure.
Dynamic Convolutional Neural Networks as Efficient Pre-trained Audio Models
Audio Spectrogram Transformers are excellent at exploiting large datasets, creating powerful pre-trained models that surpass CNNs when fine-tuned on downstream tasks.
Ranked #1 on
Instrument Recognition
on OpenMIC-2018
(using extra training data)
Towards Robust and Truly Large-Scale Audio-Sheet Music Retrieval
A range of applications of multi-modal music information retrieval is centred around the problem of connecting large collections of sheet music (images) to corresponding audio recordings, that is, identifying pairs of audio and score excerpts that refer to the same musical content.
Passage Summarization with Recurrent Models for Audio-Sheet Music Retrieval
Many applications of cross-modal music retrieval are related to connecting sheet music images to audio recordings.
Self-Supervised Contrastive Learning for Robust Audio-Sheet Music Retrieval Systems
We then conclude by arguing for the potential of self-supervised contrastive learning for alleviating the annotated data scarcity in multi-modal music retrieval models.
Symbolic Music Representations for Classification Tasks: A Systematic Evaluation
Music Information Retrieval (MIR) has seen a recent surge in deep learning-based approaches, which often involve encoding symbolic music (i. e., music represented in terms of discrete note events) in an image-like or language like fashion.
Exploring Sampling Techniques for Generating Melodies with a Transformer Language Model
To accomplish this, we train a high-capacity transformer model on a vast collection of highly-structured Irish folk melodies and analyze the musical qualities of the samples generated using distribution truncation sampling techniques.
Advancing Natural-Language Based Audio Retrieval with PaSST and Large Audio-Caption Data Sets
This work presents a text-to-audio-retrieval system based on pre-trained text and spectrogram transformers.
Ranked #4 on
Text to Audio Retrieval
on Clotho
(using extra training data)
Roman Numeral Analysis with Graph Neural Networks: Onset-wise Predictions from Note-wise Features
Roman Numeral analysis is the important task of identifying chords and their functional context in pieces of tonal music.
Predicting Music Hierarchies with a Graph-Based Neural Decoder
This paper describes a data-driven framework to parse musical sequences into dependency trees, which are hierarchical structures used in music cognition research and music analysis.
On Frequency-Wise Normalizations for Better Recording Device Generalization in Audio Spectrogram Transformers
Based on this observation, we conjecture that suppressing recording device characteristics in the input spectrogram is the most effective.
Domain Information Control at Inference Time for Acoustic Scene Classification
In the Acoustic Scene Classification task (ASC), domain shift is mainly caused by different recording devices.
Discrete Diffusion Probabilistic Models for Symbolic Music Generation
Denoising Diffusion Probabilistic Models (DDPMs) have made great strides in generating high-quality samples in both discrete and continuous domains.
Device-Robust Acoustic Scene Classification via Impulse Response Augmentation
However, we also show that DIR augmentation and Freq-MixStyle are complementary, achieving a new state-of-the-art performance on signals recorded by devices unseen during training.
Musical Voice Separation as Link Prediction: Modeling a Musical Perception Task as a Multi-Trajectory Tracking Problem
Our approach builds a graph from a musical piece, by creating one node for every note, and separates the melodic trajectories by predicting a link between two notes if they are consecutive in the same voice/stream.
Probabilistic Modelling of Signal Mixtures with Differentiable Dictionaries
We introduce a novel way to incorporate prior information into (semi-) supervised non-negative matrix factorization, which we call differentiable dictionary search.
Differentiable Dictionary Search: Integrating Linear Mixing with Deep Non-Linear Modelling for Audio Source Separation
This paper describes several improvements to a new method for signal decomposition that we recently formulated under the name of Differentiable Dictionary Search (DDS).
Learning General Audio Representations with Large-Scale Training of Patchout Audio Transformers
Furthermore, we will show that transformers trained on Audioset can be extremely effective representation extractors for a wide range of downstream tasks.
Efficient Large-scale Audio Tagging via Transformer-to-CNN Knowledge Distillation
We provide models of different complexity levels, scaling from low-complexity models up to a new state-of-the-art performance of . 483 mAP on AudioSet.
Ranked #2 on
Audio Tagging
on AudioSet
(using extra training data)
Cadence Detection in Symbolic Classical Music using Graph Neural Networks
In this work, we present a graph representation of symbolic scores as an intermediate means to solve the cadence detection task.
Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifier
Current approaches for explaining deep learning systems applied to musical data provide results in a low-level feature space, e. g., by highlighting potentially relevant time-frequency bins in a spectrogram or time-pitch bins in a piano roll.
Improved Zero-Shot Audio Tagging & Classification with Patchout Spectrogram Transformers
Standard machine learning models for tagging and classifying acoustic signals cannot handle classes that were not seen during training.
Improving Natural-Language-based Audio Retrieval with Transfer Learning and Audio & Text Augmentations
The absence of large labeled datasets remains a significant challenge in many application areas of deep learning.
Defending a Music Recommender Against Hubness-Based Adversarial Attacks
Adversarial attacks can drastically degrade performance of recommenders and other machine learning systems, resulting in an increased demand for defence mechanisms.
Fully Automatic Page Turning on Real Scores
We present a prototype of an automatic page turning system that works directly on real scores, i. e., sheet images, without any symbolic representation.
Efficient Training of Audio Transformers with Patchout
However, one of the main shortcomings of transformer models, compared to the well-established CNNs, is the computational complexity.
Ranked #3 on
Audio Classification
on FSD50K
(using extra training data)
Improving Real-time Score Following in Opera by Combining Music with Lyrics Tracking
Fully automatic opera tracking is challenging because of the acoustic complexity of the genre, combining musical and linguistic information (singing, speech) in complex ways.
Nonlinear Denoising, Linear Demixing
We cast the combinatorial problem of polyphonic piano transcription as a two stage process.
On-Line Audio-to-Lyrics Alignment Based on a Reference Performance
Audio-to-lyrics alignment has become an increasingly active research task in MIR, supported by the emergence of several open-source datasets of audio recordings with word-level lyrics annotations.
Over-Parameterization and Generalization in Audio Classification
Convolutional Neural Networks (CNNs) have been dominating classification tasks in various domains, such as machine vision, machine listening, and natural language processing.
On the Veracity of Local, Model-agnostic Explanations in Audio Classification: Targeted Investigations with Adversarial Examples
The basic idea is to identify a small set of human-understandable features of the classified example that are most influential on the classifier's prediction.
Tracing Back Music Emotion Predictions to Sound Sources and Intuitive Perceptual Qualities
In previous work, we have shown how to derive explanations of model predictions in terms of spectrogram image segments that connect to the high-level emotion prediction via a layer of easily interpretable perceptual features.
Receptive Field Regularization Techniques for Audio Classification and Tagging with Deep Convolutional Neural Networks
As state-of-the-art CNN architectures-in computer vision and other domains-tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks.
Exploiting Temporal Dependencies for Cross-Modal Music Piece Identification
This paper addresses the problem of cross-modal musical piece identification and retrieval: finding the appropriate recording(s) from a database given a sheet music query, and vice versa, working directly with audio and scanned sheet music images.
Handling Structural Mismatches in Real-time Opera Tracking
Algorithms for reliable real-time score following in live opera promise a lot of useful applications such as automatic subtitles display, or real-time video cutting in live streaming.
Multi-modal Conditional Bounding Box Regression for Music Score Following
This paper addresses the problem of sheet-image-based on-line audio-to-score alignment also known as score following.
Learning to Infer Unseen Contexts in Causal Contextual Reinforcement Learning
In Contextual Reinforcement Learning (CRL), a change in the context variable can cause a change in the distribution of the states.
Towards Explaining Expressive Qualities in Piano Recordings: Transfer of Explanatory Features via Acoustic Domain Adaptation
Emotion and expressivity in music have been topics of considerable interest in the field of music information retrieval.
Anomalous Sound Detection as a Simple Binary Classification Problem with Careful Selection of Proxy Outlier Examples
If no data with similar sounds and matching recording conditions is available, data sets with a larger diversity in these two dimensions are preferable.
Low-Complexity Models for Acoustic Scene Classification Based on Receptive Field Regularization and Frequency Damping
Deep Neural Networks are known to be very demanding in terms of computing and memory requirements.
Towards Musically Meaningful Explanations Using Source Separation
Prior work on explainable models in MIR has generally used image processing tools to produce explanations for DNN predictions, but these are not necessarily musically meaningful, or can be listened to (which, arguably, is important in music).
The Impact of Label Noise on a Music Tagger
We explore how much can be learned from noisy labels in audio music tagging.
On the Characterization of Expressive Performance in Classical Music: First Results of the Con Espressione Game
In this paper, we offer a first account of this new data resource for expressive performance research, and provide an exploratory analysis, addressing three main questions: (1) how similarly do different listeners describe a performance of a piece?
audioLIME: Listenable Explanations Using Source Separation
Deep neural networks (DNNs) are successfully applied in a wide variety of music information retrieval (MIR) tasks but their predictions are usually not interpretable.
Receptive-Field Regularized CNNs for Music Classification and Tagging
However, the MIR field is still dominated by the classical VGG-based CNN architecture variants, often in combination with more complex modules such as attention, and/or techniques such as pre-training on large datasets.
Learning to Read and Follow Music in Complete Score Sheet Images
This paper addresses the task of score following in sheet music given as unprocessed images.
On Data Augmentation and Adversarial Risk: An Empirical Analysis
Data augmentation techniques have become standard practice in deep learning, as it has been shown to greatly improve the generalisation abilities of models.
Beneath (or beyond) the surface: Discovering voice-leading patterns with skip-grams
Recurrent voice-leading patterns like the Mi-Re-Do compound cadence (MRDCC) rarely appear on the musical surface in complex polyphonic textures, so finding these patterns using computational methods remains a tremendous challenge.
Towards Reliable Real-time Opera Tracking: Combining Alignment with Audio Event Detectors to Increase Robustness
Recent advances in real-time music score following have made it possible for machines to automatically track highly complex polyphonic music, including full orchestra performances.
Emotion and Theme Recognition in Music with Frequency-Aware RF-Regularized CNNs
We present CP-JKU submission to MediaEval 2019; a Receptive Field-(RF)-regularized and Frequency-Aware CNN approach for tagging music with emotion/mood labels.
Audio-Conditioned U-Net for Position Estimation in Full Sheet Images
The goal of score following is to track a musical performance, usually in the form of audio, in a corresponding score representation.
A Study of Annotation and Alignment Accuracy for Performance Comparison in Complex Orchestral Music
We then carry out a systematic evaluation of different audio features for audio-to-audio alignment, quantifying the degree of alignment accuracy that can be achieved, and relate this to the results from the annotation study.
Receptive-field-regularized CNN variants for acoustic scene classification
One side effect of restricting the RF of CNNs is that more frequency information is lost.
Exploiting Parallel Audio Recordings to Enforce Device Invariance in CNN-based Acoustic Scene Classification
Distribution mismatches between the data seen at training and at application time remain a major challenge in all application areas of machine learning.
Towards Explainable Music Emotion Recognition: The Route via Mid-level Features
Emotional aspects play an important part in our interaction with music.
The Receptive Field as a Regularizer in Deep Convolutional Neural Networks for Acoustic Scene Classification
To this end, we analyse the receptive field (RF) of these CNNs and demonstrate the importance of the RF to the generalization capability of the models.
Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval
Quantitative and qualitative results on synthesized piano data indicate that this attention increases the robustness of the retrieval system by focusing on different parts of the input representation based on the tempo of the audio.
A Convolutional Approach to Melody Line Identification in Symbolic Scores
The backbone of the method consists of a convolutional neural network (CNN) estimating the probability that each note in the score (more precisely: each pixel in a piano roll encoding of the score) belongs to the melody line.
User Curated Shaping of Expressive Performances
Musicians produce individualized, expressive performances by manipulating parameters such as dynamics, tempo and articulation.
Two-level Explanations in Music Emotion Recognition
Current ML models for music emotion recognition, while generally working quite well, do not give meaningful or intuitive explanations for their predictions.
Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies
There has been a rapid growth of digitally available music data, including audio recordings, digitized images of sheet music, album covers and liner notes, and video clips.
Mixture Density Generative Adversarial Networks
Generative Adversarial Networks have surprising ability for generating sharp and realistic images, though they are known to suffer from the so-called mode collapse problem.
Attention as a Perspective for Learning Tempo-invariant Audio Queries
Current models for audio--sheet music retrieval via multimodal embedding space learning use convolutional neural networks with a fixed-size window for the input audio.
Automatic Chord Recognition with Higher-Order Harmonic Language Modelling
Common temporal models for automatic chord recognition model chord changes on a frame-wise basis.
Genre-Agnostic Key Classification With Convolutional Neural Networks
Finally, we investigate the model's performance on short excerpts of audio.
Improved Chord Recognition by Combining Duration and Harmonic Language Models
Chord recognition systems typically comprise an acoustic model that predicts chords for each audio frame, and a temporal model that casts these predictions into labelled chord segments.
Learning to Listen, Read, and Follow: Score Following as a Reinforcement Learning Game
Score following is the process of tracking a musical performance (audio) with respect to a known symbolic representation (a score).
A Predictive Model for Music Based on Learned Interval Representations
We show that the RGAE improves the state of the art for general connectionist sequence models in learning to predict monophonic melodies, and that ensembles of relative and absolute music processing models improve the results appreciably.
Deep SNP: An End-to-end Deep Neural Network with Attention-based Localization for Break-point Detection in SNP Array Genomic data
We show, that Deep SNP is capable of successfully predicting the presence or absence of a breakpoint in large genomic windows and outperforms state-of-the-art neural network models.
Learning Transposition-Invariant Interval Features from Symbolic Music and Audio
Many music theoretical constructs (such as scale types, modes, cadences, and chord types) are defined in terms of pitch intervals---relative distances between pitches.
Towards multi-instrument drum transcription
In this work, convolutional and convolutional recurrent neural networks are trained to transcribe a wider range of drum instruments.
Learning to Transcribe by Ear
Within this conceptual framework, the transcription process can be described as the agent interacting with the instrument in the environment, and obtaining reward by playing along with what it hears.
Investigating Label Noise Sensitivity of Convolutional Neural Networks for Fine Grained Audio Signal Labelling
We measure the effect of small amounts of systematic and random label noise caused by slightly misaligned ground truth labels in a fine grained audio signal labeling task.
A Large-Scale Study of Language Models for Chord Prediction
We conduct a large-scale study of language models for chord prediction.
Deep Within-Class Covariance Analysis for Robust Audio Representation Learning
To tackle this problem, we propose Deep Within-Class Covariance Analysis (DWCCA), a deep neural network layer that significantly reduces the within-class covariance of a DNN's representation, improving performance on unseen test data from a shifted distribution.
What were you expecting? Using Expectancy Features to Predict Expressive Performances of Classical Piano Music
In this paper we present preliminary work examining the relationship between the formation of expectations and the realization of musical performances, paying particular attention to expressive tempo and dynamics.
Learning Musical Relations using Gated Autoencoders
Music is usually highly structured and it is still an open question how to design models which can successfully learn to recognize and represent musical structure.
Probabilistic Generative Adversarial Networks
The central idea is to integrate a probabilistic model (a Gaussian Mixture Model, in our case) into the GAN framework which supports a new kind of loss function (based on likelihood rather than classification loss), and at the same time gives a meaningful measure of the quality of the outputs generated by the network.
Likelihood Estimation for Generative Adversarial Networks
We present a simple method for assessing the quality of generated images in Generative Adversarial Networks (GANs).
A Hybrid Approach with Multi-channel I-Vectors and Convolutional Neural Networks for Acoustic Scene Classification
Finally, we propose a hybrid system for ASC using multi-channel i-vectors and CNNs by utilizing a score fusion technique.
End-to-End Musical Key Estimation Using a Convolutional Neural Network
We present an end-to-end system for musical key estimation, based on a convolutional neural network.
On the Futility of Learning Complex Frame-Level Language Models for Chord Recognition
Chord recognition systems use temporal models to post-process frame-wise chord preditions from acoustic models.
On the Potential of Simple Framewise Approaches to Piano Transcription
In an attempt at exploring the limitations of simple approaches to the task of piano transcription (as usually defined in MIR), we conduct an in-depth analysis of neural network-based framewise transcription.
Feature Learning for Chord Recognition: The Deep Chroma Extractor
We explore frame-level audio feature learning for chord recognition using artificial neural networks.
Towards Score Following in Sheet Music Images
This paper addresses the matching of short music audio snippets to the corresponding pixel location in images of sheet music.
Towards End-to-End Audio-Sheet-Music Retrieval
This paper demonstrates the feasibility of learning to retrieve short snippets of sheet music (images) when given a short query excerpt of music (audio) -- and vice versa --, without any symbolic representation of music or scores.
A Fully Convolutional Deep Auditory Model for Musical Chord Recognition
We show that the learned auditory system extracts musically interpretable features, and that the proposed chord recognition system achieves results on par or better than state-of-the-art algorithms.
Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints
We introduce a method for imposing higher-level structure on generated, polyphonic music.
Deep Linear Discriminant Analysis
The central idea of this paper is to put LDA on top of a deep neural network.
