Search Results for author:
Found 50 papers, 35 papers with code
YuE: Scaling Open Foundation Models for Long-Form Music Generation
We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the LLaMA2 architecture.
Language Model Mapping in Multimodal Music Learning: A Grand Challenge Proposal
We have seen remarkable success in representation learning and language models (LMs) using deep neural networks.
Music for All: Representational Bias and Cross-Cultural Adaptability of Music Generation Models
The advent of Music-Language Models has greatly enhanced the automatic music generation capability of AI systems, but they are also limited in their coverage of the musical genres and cultures of the world.
Exploring GPT's Ability as a Judge in Music Understanding
We convert the music data to symbolic inputs and evaluate LLMs' ability in detecting annotation errors in three key MIR tasks: beat tracking, chord extraction, and key estimation.
M2M-Gen: A Multimodal Framework for Automated Background Music Generation in Japanese Manga Using Large Language Models
To address these challenges, we propose an automated music generation pipeline that produces background music for an input manga book.
CalliffusionV2: Personalized Natural Calligraphy Generation with Flexible Multi-modal Control
In this paper, we introduce CalliffusionV2, a novel system designed to produce natural Chinese calligraphy with flexible multi-modal control.
Unifying Multitrack Music Arrangement via Reconstruction Fine-Tuning and Efficient Tokenization
Automatic music arrangement streamlines the creation of musical variants for composers and arrangers, reducing reliance on extensive music expertise.
Foundation Models for Music: A Survey
In recent years, foundation models (FMs) such as large language models (LLMs) and latent diffusion models (LDMs) have profoundly impacted diverse sectors, including music.
Emergent Interpretable Symbols and Content-Style Disentanglement via Variance-Invariance Constraints
We contribute an unsupervised method that effectively learns from raw observation and disentangles its latent space into content and style representations.
Proceedings of The second international workshop on eXplainable AI for the Arts (XAIxArts)
This second international workshop on explainable AI for the Arts (XAIxArts) brought together a community of researchers in HCI, Interaction Design, AI, explainable AI (XAI), and digital arts to explore the role of XAI for the Arts.
Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuning
Recent advances in text-to-music editing, which employ text queries to modify music (e. g.\ by changing its style or adjusting instrumental components), present unique challenges and opportunities for AI-assisted music creation.
Human-Centered LLM-Agent User Interface: A Position Paper
Large Language Model (LLM) -in-the-loop applications have been shown to effectively interpret the human user's commands, make plans, and operate external tools/systems accordingly.
Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models
A cascaded diffusion model is trained to model the hierarchical language, where each level is conditioned on its upper levels.
MuPT: A Generative Symbolic Music Pretrained Transformer
In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music.
ChatMusician: Understanding and Generating Music Intrinsically with LLM
It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language.
Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls
This approach enables autoregressive language models to seamlessly address music inpainting tasks.
MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
This paper introduces a novel approach to the editing of music generated by such models, enabling the modification of specific attributes, such as genre, mood and instrument, while maintaining other aspects unchanged.
CalliPaint: Chinese Calligraphy Inpainting with Diffusion Model
Chinese calligraphy can be viewed as a unique form of visual art.
Content-based Controls For Music Large Language Modeling
We aim to further equip the models with direct and content-based controls on innate music languages such as pitch, chords and drum track.
Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
In the realm of music AI, arranging rich and structured multi-track accompaniments from a simple lead sheet presents significant challenges.
Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing
Creating music is iterative, requiring varied methods at each stage.
Motif-Centric Representation Learning for Symbolic Music
Music motif, as a conceptual building block of composition, is crucial for music structure analysis and automatic composition.
Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls
We propose Polyffusion, a diffusion model that generates polyphonic music scores by regarding music as image-like piano roll representations.
On the Effectiveness of Speech Self-supervised Learning for Music
Our findings suggest that training with music data can generally improve performance on MIR tasks, even when models are trained using paradigms designed for speech.
MARBLE: Music Audio Representation Benchmark for Universal Evaluation
This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark.
MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training
Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored.
Calliffusion: Chinese Calligraphy Generation and Style Transfer with Diffusion Modeling
In this paper, we propose Calliffusion, a system for generating high-quality Chinese calligraphy using diffusion models.
Learning Interpretable Low-dimensional Representation via Physical Symmetry
We have recently seen great progress in learning interpretable music representations, ranging from basic factors, such as pitch and timbre, to high-level concepts, such as chord and texture.
Vis2Mus: Exploring Multimodal Representation Mapping for Controllable Music Generation
In this study, we explore the representation mapping from the domain of visual arts to the domain of music, with which we can use visual arts as an effective handle to control music generation.
Self-Supervised Hierarchical Metrical Structure Modeling
We propose a novel method to model hierarchical metrical structures for both symbolic music and audio signals in a self-supervised manner with minimal domain knowledge.
Modeling Perceptual Loudness of Piano Tone: Theory and Applications
The relationship between perceptual loudness and physical attributes of sound is an important subject in both computer music and psychoacoustics.
Learning Hierarchical Metrical Structure Beyond Measures
In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores.
AccoMontage2: A Complete Harmonization and Accompaniment Arrangement System
We propose AccoMontage2, a system capable of doing full-length song harmonization and accompaniment arrangement based on a lead melody.
Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model
Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives.
Learning long-term music representations via hierarchical contextual constraints
Learning symbolic music representations, especially disentangled representations with probabilistic interpretations, has been shown to benefit both music understanding and generation.
Audio-to-symbolic Arrangement via Cross-modal Music Representation Learning
This is the audio-to-symbolic arrangement problem we tackle in this paper.
AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer
Accompaniment arrangement is a difficult music generation task involving intertwined constraints of melody, harmony, texture, and music structure.
A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis
We propose a unified model for three inter-related tasks: 1) to \textit{separate} individual sound sources from a mixed music audio, 2) to \textit{transcribe} each sound source to MIDI notes, and 3) to\textit{ synthesize} new pieces based on the timbre of separated sources.
Learning Interpretable Representation for Controllable Polyphonic Music Generation
While deep generative models have become the leading methods for algorithmic composition, it remains a challenging problem to control the generation process because the latent variables of most deep-learning models lack good interpretability.
PIANOTREE VAE: Structured Representation Learning for Polyphonic Music
The dominant approach for music representation learning involves the deep unsupervised model family variational autoencoder (VAE).
POP909: A Pop-song Dataset for Music Arrangement Generation
The main body of the dataset contains the vocal melody, the lead instrument melody, and the piano accompaniment for each song in MIDI format, which are aligned to the original audio files.
Word Representation for Rhythms
This paper proposes a word representation strategy for rhythm patterns.
Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions
Automatic music generation is an interdisciplinary research topic that combines computational creativity and semantic analysis of music to create automatic machine improvisations.
Deep Music Analogy Via Latent Representation Disentanglement
Analogy-making is a key method for computer algorithms to generate both natural and creative music pieces.
Inspecting and Interacting with Meaningful Music Representations using VAE
Variational Autoencoders(VAEs) have already achieved great results on image generation and recently made promising progress on music generation.
A Framework for Automated Pop-song Melody Generation with Piano Accompaniment Arrangement
Second, the melody generation model generates the lead melody and other voices (melody lines) of the accompaniment using seasonal ARMA (Autoregressive Moving Average) processes.
The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generation
With recent breakthroughs in artificial neural networks, deep generative models have become one of the leading techniques for computational creativity.
Melodic Phrase Segmentation By Deep Neural Networks
Automated melodic phrase detection and segmentation is a classical task in content-based music information retrieval and also the key towards automated music structure analysis.
