Search Results for author:
Found 19 papers, 10 papers with code
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives
Multi-modal music generation, using multiple modalities like text, images, and video alongside musical scores and audio as guidance, is an emerging research area with broad applications.
A Comprehensive Survey on Generative AI for Video-to-Music Generation
The burgeoning growth of video-to-music generation can be attributed to the ascendancy of multimodal generative models.
SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training
In this paper, we propose SongGLM, a lyric-to-melody generation system that leverages 2D alignment encoding and multi-task pre-training based on the General Language Model (GLM) to guarantee the alignment and harmony between lyrics and melodies.
Risk of Text Backdoor Attacks Under Dataset Distillation
We propose a framework for backdoor attacks in the context of text dataset distillation, termed Text Backdoor Attack under Dataset Distillation (TBADD).
MetaBGM: Dynamic Soundtrack Transformation For Continuous Multi-Scene Experiences With Ambient Awareness And Personalization
This paper introduces MetaBGM, a groundbreaking framework for generating background music that adapts to dynamic scenes and real-time user interactions.
SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancement
Singing voice conversion (SVC) aims to convert a singer's voice to another singer's from a reference audio while keeping the original semantics.
MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generation
This task is aimed at bridging the gap between colloquial language understanding and auditory expression within an AI model, with the ultimate goal of creating songs that accurately satisfy human auditory expectations and structurally align with musical norms.
MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music
To this end, we present MuChin, the first open-source music description benchmark in Chinese colloquial language, designed to evaluate the performance of multimodal LLMs in understanding and describing music.
End-to-end Learnable Clustering for Intent Learning in Recommendation
Concretely, we encode user behavior sequences and initialize the cluster centers (latent intents) as learnable neurons.
MelodyGLM: Multi-task Pre-training for Symbolic Melody Generation
In particular, subjective evaluations show that, on the melody continuation task, MelodyGLM gains average improvements of 0. 82, 0. 87, 0. 78, and 0. 94 in consistency, rhythmicity, structure, and overall quality, respectively.
REMAST: Real-time Emotion-based Music Arrangement with Soft Transition
Music as an emotional intervention medium has important applications in scenarios such as music therapy, games, and movies.
WuYun: Exploring hierarchical skeleton-guided melody generation using knowledge-enhanced deep learning
Although deep learning has revolutionized music generation, existing methods for structured melody generation follow an end-to-end left-to-right note-by-note generative paradigm and treat each note equally.
SongDriver: Real-time Music Accompaniment Generation without Logical Latency nor Exposure Bias
In this paper, we propose SongDriver, a real-time music accompaniment generation system without logical latency nor exposure bias.
Automatic Song Translation for Tonal Languages
This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning.
S3T: Self-Supervised Pre-training with Swin Transformer for Music Classification
To our knowledge, S3T is the first method combining the Swin Transformer with a self-supervised learning method for music classification.
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e. g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i. e., the system consists of a lyric-to-template module and a template-to-melody module).
PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription
Considering that there is a large amount of ASR training data, a straightforward method is to leverage ASR data to enhance ALT training.
Automatic Lyrics Transcription
Automatic Speech Recognition
+3
Denoising Text to Speech with Frame-Level Noise Modeling
In DenoiSpeech, we handle real-world noisy speech by modeling the fine-grained frame-level noise with a noise condition module, which is jointly trained with the TTS model.
A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals
In this paper, our work makes an attempt to fuse the subject individual EDA features and the external evoked music features.
