Search Results for author:
Found 61 papers, 33 papers with code
JamendoMaxCaps: A Large Scale Music-caption Dataset with Imputed Metadata
We introduce JamendoMaxCaps, a large-scale music-caption dataset featuring over 200, 000 freely licensed instrumental tracks from the renowned Jamendo platform.
Towards Unified Music Emotion Recognition across Dimensional and Categorical Models
Moreover, knowledge distillation is employed to transfer the knowledge of teacher models trained on individual datasets to a student model, enhancing its ability to generalize across multiple tasks.
ImprovNet: Generating Controllable Musical Improvisations with Iterative Corruption Refinement
However, in the realm of symbolic music, generating controllable and expressive performance-level style transfers for complete musical works remains challenging due to limited datasets, especially for genres such as jazz, and the lack of unified models that can handle multiple music generation tasks.
Text2midi: Generating Symbolic Music from Captions
This paper introduces text2midi, an end-to-end model to generate MIDI files from textual descriptions.
MIRFLEX: Music Information Retrieval Feature Library for Extraction
This paper introduces an extendable modular system that compiles a range of music feature extraction models to aid music information retrieval research.
DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech
Recent advancements in Text-to-Speech (TTS) systems have enabled the generation of natural and expressive speech from textual input.
Leveraging LLM Embeddings for Cross Dataset Label Alignment and Zero Shot Music Emotion Prediction
In this work, we present a novel method for music emotion recognition that leverages Large Language Model (LLM) embeddings for label alignment across multiple datasets and zero-shot prediction on novel categories.
Prevailing Research Areas for Music AI in the Era of Foundation Models
In tandem with the recent advancements in foundation model research, there has been a surge of generative music AI applications within the past few years.
PRESENT: Zero-Shot Text-to-Prosody Control
We attain 25. 3% hanzi CER and 13. 0% pinyin CER with the JETS model.
BandControlNet: Parallel Transformers-based Steerable Popular Music Generation with Fine-Grained Spatiotemporal Features
Subsequently, we release BandControlNet, a conditional model based on parallel Transformers, to tackle the multiple music sequences and generate high-quality music samples that are conditioned to the given spatiotemporal control features.
DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguage
Laughing, sighing, stuttering, and other forms of paralanguage do not contribute any direct lexical meaning to speech, but they provide crucial propositional context that aids semantic and pragmatic processes such as irony.
DeepUnifiedMom: Unified Time-series Momentum Portfolio Construction via Multi-Task Learning with Multi-Gate Mixture of Experts
This paper introduces DeepUnifiedMom, a deep learning framework that enhances portfolio management through a multi-task learning approach and a multi-gate mixture of experts.
Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges
Deep learning models for music have advanced drastically in recent years, but how good are machine learning models at capturing emotion, and what challenges are researchers facing?
MidiCaps: A large-scale MIDI dataset with text captions
This work aims to enable research that combines LLMs with symbolic music by presenting, the first openly available large-scale MIDI dataset with text captions.
Accent Conversion in Text-To-Speech Using Multi-Level VAE and Adversarial Training
With rapid globalization, the need to build inclusive and representative speech technology cannot be overstated.
Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey
Music has been frequently compared to language, as they share several similarities, including sequential representations of text and music.
Mustango: Toward Controllable Text-to-Music Generation
Through extensive experiments, we show that the quality of the music generated by Mustango is state-of-the-art, and the controllability through music-specific text prompts greatly outperforms other models such as MusicGen and AudioLDM2.
Ranked #1 on
Text-to-Music Generation
on MusicBench
Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model
These distinct features are then employed as guiding input to our music generation model.
Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning
The performance of existing TSMOM strategies, however, relies not only on the quality of the momentum signal but also on the efficacy of the volatility estimator.
Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training
Jointist consists of an instrument recognition module that conditions the other two modules: a transcription module that outputs instrument-specific piano rolls, and a source separation module that utilizes instrument information and transcription results.
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech
Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary.
Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder
Accent plays a significant role in speech communication, influencing one's capability to understand as well as conveying a person's identity.
DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT).
Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models
Our results show that the model outperforms existing state-of-the-art models when forecasting extreme volatility spikes for Bitcoin using CryptoQuant data as well as whale-alert tweets.
Jointist: Joint Learning for Multi-instrument Transcription and Its Applications
However, its novelty necessitates a new perspective on how to evaluate such a model.
Ranked #4 on
Music Transcription
on Slakh2100
PreBit -- A multimodal model with Twitter FinBERT embeddings for extreme price movement prediction of Bitcoin
In our hybrid model, we use sentence-level FinBERT embeddings, pretrained on financial lexicons, so as to capture the full contents of the tweets and feed it to the model in an understandable way.
HEAR: Holistic Evaluation of Audio Representations
The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios.
Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses
In this paper we present MusicVideos (MuVi), a novel dataset for affective multimedia content analysis to study how the auditory and visual modalities contribute to the perceived emotion of media.
MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control
We present a novel music generation framework for music infilling, with a user friendly interface.
Conditional Drums Generation using Compound Word Representations
The field of automatic music composition has seen great progress in recent years, specifically with the invention of transformer-based architectures.
AttendAffectNet–Emotion Prediction of Movie Viewers Using Multimodal Fusion with Self-Attention
The models that use all visual, audio, and text features simultaneously as their inputs performed better than those using features extracted from each modality separately.
ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data
Most of the current supervised automatic music transcription (AMT) models lack the ability to generalize.
aiSTROM -- A roadmap for developing a successful AI strategy
This provides a unique and integrated approach that guides managers and lead developers through the various challenges in the implementation process.
Deep Neural Network Based Respiratory Pathology Classification Using Cough Sounds
The resulting trained model when trained for classifying two classes of coughs -- healthy or pathology (in general or belonging to a specific respiratory pathology), reaches accuracy exceeding 84\% when classifying cough to the label provided by the physicians' diagnosis.
Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework
In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts.
Underwater Acoustic Communication Receiver Using Deep Belief Network
Our proposed receiver system comprises of DBN based de-noising and classification of the received signal.
AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies
In this work, we propose different variants of the self-attention based network for emotion prediction from movies, which we call AttendAffectNet.
The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracy
We attempt to use only the pitch labels (together with spectrogram reconstruction loss) and explore how far this model can go without introducing supervised sub-tasks.
Hit Song Prediction Based on Early Adopter Data and Audio Features
Billions of USD are invested in new artists and songs by the music industry every year.
A variational autoencoder for music generation controlled by tonal tension
Many of the music generation systems based on neural networks are fully autonomous and do not offer control over the generation process.
Sound Symbolic Computation Audio and Speech Processing
A dataset and classification model for Malay, Hindi, Tamil and Chinese music
We use this new dataset to train different classification models to distinguish the origin of the music in terms of these ethnic groups.
Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling
Using arousal as an example of a high-level feature, we show that the "faders" of our model are disentangled and change linearly w. r. t.
PerceptionGAN: Real-world Image Construction from Provided Text through Perceptual Understanding
This initial image is then improved by conditioning on the text.
Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance
We present a controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE), which can generate realistic piano performances in the audio domain that closely follows temporal conditions of two essential style features for piano performances: articulation and dynamics.
Acoustic prediction of flowrate: varying liquid jet stream onto a free surface
Information on liquid jet stream flow is crucial in many real world applications.
The impact of Audio input representations on neural network based music transcription
This paper thoroughly analyses the effect of different input representations on polyphonic multi-instrument music transcription.
Sound Audio and Speech Processing
nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networks
First, it takes a lot of hard disk space to store different frequency domain representations.
Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders
We propose a flexible framework that deals with both singer conversion and singers vocal technique conversion.
Midi Miner -- A Python library for tonal tension and track classification
We present a Python library, called Midi Miner, that can calculate tonal tension and classify different tracks.
Latent space representation for multi-target speaker detection and identification with a sparse dataset using Triplet neural networks
When reducing the training data to only using the train set, our method results in 309 confusions for the Multi-target speaker identification task, which is 46% better than the baseline model.
Multimodal Deep Models for Predicting Affective Responses Evoked by Movies
Interestingly, we also observe that the optical flow is more informative than the RGB in videos, and overall, models using audio features are more accurate than those based on video features when making the final prediction of evoked emotions.
Doppler Invariant Demodulation for Shallow Water Acoustic Communications Using Deep Belief Networks
The proposed method comprises of a ML based feature extraction method and classification technique.
Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders
Specifically, we use two separate encoders to learn distinct latent spaces for timbre and pitch, which form Gaussian mixture components representing instrument identity and pitch, respectively.
Towards robust audio spoofing detection: a detailed comparison of traditional and learned features
Finally, we evaluate the performance of our robust replay speaker detection system with a wide variety and different combinations of both extracted and machine learned audio features on the `out in the wild' ASVspoof 2017 dataset.
Dance Hit Song Prediction
Record companies invest billions of dollars in new talent around the globe each year.
MorpheuS: generating structured music with constrained patterns and tension
MorpheuS' novel framework has the ability to generate polyphonic pieces with a given tension profile and long- and short-term repeated pattern structures.
Sound Audio and Speech Processing
A Functional Taxonomy of Music Generation Systems
Digital advances have transformed the face of automatic music generation since its beginnings at the dawn of computing.
Singing Voice Separation Using a Deep Convolutional Neural Network Trained by Ideal Binary Mask and Cross Entropy
We present a unique neural network approach inspired by a technique that has revolutionized the field of vision: pixel-wise image classification, which we combine with cross entropy loss and pretraining of the CNN as an autoencoder on singing voice spectrograms.
From Context to Concept: Exploring Semantic Relationships in Music with Word2Vec
In this newly learned vector space, a metric based on cosine distance is able to distinguish between functional chord relationships, as well as harmonic associations in the music.
Modeling Musical Context with Word2vec
A visualization of the reduced vector space using t-distributed stochastic neighbor embedding shows that the resulting embedded vector space captures tonal relationships, even without any explicit information about the musical contents of the slices.
Proceedings of the First International Workshop on Deep Learning and Music
Proceedings of the First International Workshop on Deep Learning and Music, joint with IJCNN, Anchorage, US, May 17-18, 2017
