Search Results for author:
Found 46 papers, 23 papers with code
Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey
Music has been frequently compared to language, as they share several similarities, including sequential representations of text and music.
Mustango: Toward Controllable Text-to-Music Generation
Through extensive experiments, we show that the quality of the music generated by Mustango is state-of-the-art, and the controllability through music-specific text prompts greatly outperforms other models such as MusicGen and AudioLDM2.
Ranked #1 on
Text-to-Music Generation
on MusicBench
Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model
These distinct features are then employed as guiding input to our music generation model.
Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning
The performance of existing TSMOM strategies, however, relies not only on the quality of the momentum signal but also on the efficacy of the volatility estimator.
Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training
Jointist consists of an instrument recognition module that conditions the other two modules: a transcription module that outputs instrument-specific piano rolls, and a source separation module that utilizes instrument information and transcription results.
SNIPER Training: Variable Sparsity Rate Training For Text-To-Speech
Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary.
Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder
Accent plays a significant role in speech communication, influencing understanding capabilities and also conveying a person's identity.
DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT).
Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models
Our results show that the model outperforms existing state-of-the-art models when forecasting extreme volatility spikes for Bitcoin using CryptoQuant data as well as whale-alert tweets.
Jointist: Joint Learning for Multi-instrument Transcription and Its Applications
However, its novelty necessitates a new perspective on how to evaluate such a model.
Ranked #1 on
Music Transcription
on Slakh2100
PreBit -- A multimodal model with Twitter FinBERT embeddings for extreme price movement prediction of Bitcoin
In our hybrid model, we use sentence-level FinBERT embeddings, pretrained on financial lexicons, so as to capture the full contents of the tweets and feed it to the model in an understandable way.
HEAR: Holistic Evaluation of Audio Representations
The aim of the HEAR benchmark is to develop a general-purpose audio representation that provides a strong basis for learning in a wide variety of tasks and scenarios.
Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses
In this paper we present MusicVideos (MuVi), a novel dataset for affective multimedia content analysis to study how the auditory and visual modalities contribute to the perceived emotion of media.
MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control
We present a novel music generation framework for music infilling, with a user friendly interface.
Conditional Drums Generation using Compound Word Representations
The field of automatic music composition has seen great progress in recent years, specifically with the invention of transformer-based architectures.
AttendAffectNet–Emotion Prediction of Movie Viewers Using Multimodal Fusion with Self-Attention
The models that use all visual, audio, and text features simultaneously as their inputs performed better than those using features extracted from each modality separately.
ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data
Most of the current supervised automatic music transcription (AMT) models lack the ability to generalize.
aiSTROM -- A roadmap for developing a successful AI strategy
This provides a unique and integrated approach that guides managers and lead developers through the various challenges in the implementation process.
Deep Neural Network Based Respiratory Pathology Classification Using Cough Sounds
The resulting trained model when trained for classifying two classes of coughs -- healthy or pathology (in general or belonging to a specific respiratory pathology), reaches accuracy exceeding 84\% when classifying cough to the label provided by the physicians' diagnosis.
Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework
In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts.
Underwater Acoustic Communication Receiver Using Deep Belief Network
Our proposed receiver system comprises of DBN based de-noising and classification of the received signal.
AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies
In this work, we propose different variants of the self-attention based network for emotion prediction from movies, which we call AttendAffectNet.
The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracy
We attempt to use only the pitch labels (together with spectrogram reconstruction loss) and explore how far this model can go without introducing supervised sub-tasks.
Hit Song Prediction Based on Early Adopter Data and Audio Features
Billions of USD are invested in new artists and songs by the music industry every year.
A variational autoencoder for music generation controlled by tonal tension
Many of the music generation systems based on neural networks are fully autonomous and do not offer control over the generation process.
Sound Symbolic Computation Audio and Speech Processing
A dataset and classification model for Malay, Hindi, Tamil and Chinese music
We use this new dataset to train different classification models to distinguish the origin of the music in terms of these ethnic groups.
Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling
Using arousal as an example of a high-level feature, we show that the "faders" of our model are disentangled and change linearly w. r. t.
PerceptionGAN: Real-world Image Construction from Provided Text through Perceptual Understanding
This initial image is then improved by conditioning on the text.
Acoustic prediction of flowrate: varying liquid jet stream onto a free surface
Information on liquid jet stream flow is crucial in many real world applications.
Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance
We present a controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE), which can generate realistic piano performances in the audio domain that closely follows temporal conditions of two essential style features for piano performances: articulation and dynamics.
The impact of Audio input representations on neural network based music transcription
This paper thoroughly analyses the effect of different input representations on polyphonic multi-instrument music transcription.
Sound Audio and Speech Processing
nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networks
First, it takes a lot of hard disk space to store different frequency domain representations.
Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders
We propose a flexible framework that deals with both singer conversion and singers vocal technique conversion.
Midi Miner -- A Python library for tonal tension and track classification
We present a Python library, called Midi Miner, that can calculate tonal tension and classify different tracks.
Latent space representation for multi-target speaker detection and identification with a sparse dataset using Triplet neural networks
When reducing the training data to only using the train set, our method results in 309 confusions for the Multi-target speaker identification task, which is 46% better than the baseline model.
Multimodal Deep Models for Predicting Affective Responses Evoked by Movies
Interestingly, we also observe that the optical flow is more informative than the RGB in videos, and overall, models using audio features are more accurate than those based on video features when making the final prediction of evoked emotions.
Doppler Invariant Demodulation for Shallow Water Acoustic Communications Using Deep Belief Networks
The proposed method comprises of a ML based feature extraction method and classification technique.
Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders
Specifically, we use two separate encoders to learn distinct latent spaces for timbre and pitch, which form Gaussian mixture components representing instrument identity and pitch, respectively.
Towards robust audio spoofing detection: a detailed comparison of traditional and learned features
Finally, we evaluate the performance of our robust replay speaker detection system with a wide variety and different combinations of both extracted and machine learned audio features on the `out in the wild' ASVspoof 2017 dataset.
Dance Hit Song Prediction
Record companies invest billions of dollars in new talent around the globe each year.
MorpheuS: generating structured music with constrained patterns and tension
MorpheuS' novel framework has the ability to generate polyphonic pieces with a given tension profile and long- and short-term repeated pattern structures.
Sound Audio and Speech Processing
A Functional Taxonomy of Music Generation Systems
Digital advances have transformed the face of automatic music generation since its beginnings at the dawn of computing.
Singing Voice Separation Using a Deep Convolutional Neural Network Trained by Ideal Binary Mask and Cross Entropy
We present a unique neural network approach inspired by a technique that has revolutionized the field of vision: pixel-wise image classification, which we combine with cross entropy loss and pretraining of the CNN as an autoencoder on singing voice spectrograms.
From Context to Concept: Exploring Semantic Relationships in Music with Word2Vec
In this newly learned vector space, a metric based on cosine distance is able to distinguish between functional chord relationships, as well as harmonic associations in the music.
Modeling Musical Context with Word2vec
A visualization of the reduced vector space using t-distributed stochastic neighbor embedding shows that the resulting embedded vector space captures tonal relationships, even without any explicit information about the musical contents of the slices.
Proceedings of the First International Workshop on Deep Learning and Music
Proceedings of the First International Workshop on Deep Learning and Music, joint with IJCNN, Anchorage, US, May 17-18, 2017
