Search Results for author:
Found 8 papers, 6 papers with code
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
By enhancing the semantic ability of the codec, X-Codec significantly reduces WER in speech synthesis tasks and extends these benefits to non-speech applications, including music and sound generation.
MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions
To fulfill this gap, we present MMTrail, a large-scale multi-modality video-language dataset incorporating more than 20M trailer clips with visual captions, and 2M high-quality clips with multimodal captions.
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling
In this work, we systematically study music generation conditioned solely on the video.
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild
Here, we construct the audio ControlNet through a trainable copy of our pre-trained diffusion model.
ComposerX: Multi-Agent Symbolic Music Composition with LLMs
Music composition represents the creative side of humanity, and itself is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints.
FlashSpeech: Efficient Zero-Shot Speech Synthesis
The generation processes of FlashSpeech can be achieved efficiently with one or two sampling steps while maintaining high audio quality and high similarity to the audio prompt for zero-shot speech generation.
Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation
In addition, the lack of large-scale available datasets with emotional transition speech and corresponding 3D human gestures also limits the addressing of this task.
LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT
We introduce LyricWhiz, a robust, multilingual, and zero-shot automatic lyrics transcription method achieving state-of-the-art performance on various lyrics transcription datasets, even in challenging genres such as rock and metal.
