Search Results for author:
Found 7 papers, 1 papers with code
Enhancing Real-World Active Speaker Detection with Multi-Modal Extraction Pre-Training
Audio-visual active speaker detection (AV-ASD) aims to identify which visible face is speaking in a scene with one or more persons.
Text-guided HuBERT: Self-Supervised Speech Pre-training via Generative Adversarial Networks
In this paper, we investigate a new way to pre-train such a joint speech-text model to learn enhanced speech representations and benefit various speech-related downstream tasks.
Self-Transriber: Few-shot Lyrics Transcription with Self-training
The current lyrics transcription approaches heavily rely on supervised learning with labeled data, but such data are scarce and manual labeling of singing is expensive.
token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Text
Firstly, due to the distinct characteristics between speech and text modalities, where speech is continuous while text is discrete, we first discretize speech into a sequence of discrete speech tokens to solve the modality mismatch problem.
PoLyScriber: Integrated Fine-tuning of Extractor and Lyrics Transcriber for Polyphonic Music
Lyrics transcription of polyphonic music is challenging as the background music affects lyrics intelligibility.
Music-robust Automatic Lyrics Transcription of Polyphonic Music
To improve the robustness of lyrics transcription to the background music, we propose a strategy of combining the features that emphasize the singing vocals, i. e. music-removed features that represent singing vocal extracted features, and the features that capture the singing vocals as well as the background music, i. e. music-present features.
Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Music
Lyrics transcription of polyphonic music is challenging not only because the singing vocals are corrupted by the background music, but also because the background music and the singing style vary across music genres, such as pop, metal, and hip hop, which affects lyrics intelligibility of the song in different ways.
