Search Results for author:
Found 6 papers, 4 papers with code
SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data
Second, we propose a new hybrid architecture that merges the cascaded and parallel architectures of SpeechCLIP into a multi-task learning framework.
Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model
Recent advances in self-supervised speech models have shown significant improvement in many downstream tasks.
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Audio-visual representation learning aims to develop systems with human-like perception by utilizing correlation between auditory and visual information.
M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval
This work investigates the use of large-scale, English-only pre-trained models (CLIP and HuBERT) for multilingual image-speech retrieval.
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Data-driven speech processing models usually perform well with a large amount of text supervision, but collecting transcribed speech data is costly.
Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer
To condition the generation process of such a model with a user-specified sequence, a popular approach is to take that conditioning sequence as a priming sequence and ask a Transformer decoder to generate a continuation.
Music Generation
Representation Learning
Sound
Multimedia
Audio and Speech Processing
