Search Results for author:
Found 15 papers, 3 papers with code
Accompanied Singing Voice Synthesis with Fully Text-controlled Melody
Text-to-song (TTSong) is a music generation task that synthesizes accompanied singing voices.
Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion
We adopt discrete-unit random resampling and pitch corruption strategies, enabling training with unpaired singing data and thus mitigating the issue of data scarcity.
Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching
By employing a non-autoregressive vector field estimator based on a feed-forward transformer and channel-level cross-modal feature fusion with strong temporal alignment, our model generates audio that is highly synchronized with the input video.
Ranked #4 on
Video-to-Sound Generation
on VGG-Sound
Robust Singing Voice Transcription Serves Synthesis
Note-level Automatic Singing Voice Transcription (AST) converts singing recordings into note sequences, facilitating the automatic annotation of singing datasets for Singing Voice Synthesis (SVS) applications.
Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment
A song is a combination of singing voice and accompaniment.
AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contexts
Leveraging the synergy of both audio data and visual data is essential for understanding human emotions and behaviors, especially in in-the-wild setting.
Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimation
This paper presents our approach for the VA (Valence-Arousal) estimation task in the ABAW6 competition.
Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modeling
Facial Expression Recognition (FER) plays a crucial role in computer vision and finds extensive applications across various fields.
Facial Expression Recognition
Facial Expression Recognition (FER)
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Recent singing-voice-synthesis (SVS) methods have achieved remarkable audio quality and naturalness, yet they lack the capability to control the style attributes of the synthesized singing explicitly.
Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning
This paper proposes a novel framework for multi-label image recognition without any training data, called data-free framework, which uses knowledge of pre-trained Large Language Model (LLM) to learn prompts to adapt pretrained Vision-Language Model (VLM) like CLIP to multilabel classification.
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer
The acoustic language model we introduce for style transfer leverages self-supervised in-context learning, acquiring style transfer ability without relying on any speaker-parallel data, thereby overcoming data scarcity.
Make-A-Voice: Unified Voice Synthesis With Discrete Representation
Various applications of voice synthesis have been developed independently despite the fact that they generate "voice" as output in common.
Connecting Multi-modal Contrastive Representations
This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR).
FastLTS: Non-Autoregressive End-to-End Unconstrained Lip-to-Speech Synthesis
To tackle these problems, we propose FastLTS, a non-autoregressive end-to-end model which can directly synthesize high-quality speech audios from unconstrained talking videos with low latency, and has a relatively small model size.
