Search Results for author:
Found 10 papers, 5 papers with code
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models.
Measuring Sound Symbolism in Audio-visual Models
Audio-visual pre-trained models have gained substantial attention recently and demonstrated superior performance on various audio-visual tasks.
Self-supervised Speech Models for Word-Level Stuttered Speech Detection
In this study, we curated a stuttered speech dataset with word-level annotations and introduced a word-level stuttering speech detection model leveraging self-supervised speech models.
Interface Design for Self-Supervised Speech Models
In particular, we show that a convolutional interface whose depth scales logarithmically with the depth of the upstream model consistently outperforms many other interface designs.
SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data
Second, we propose a new hybrid architecture that merges the cascaded and parallel architectures of SpeechCLIP into a multi-task learning framework.
Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model
Recent advances in self-supervised speech models have shown significant improvement in many downstream tasks.
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Audio-visual representation learning aims to develop systems with human-like perception by utilizing correlation between auditory and visual information.
M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval
This work investigates the use of large-scale, English-only pre-trained models (CLIP and HuBERT) for multilingual image-speech retrieval.
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Data-driven speech processing models usually perform well with a large amount of text supervision, but collecting transcribed speech data is costly.
Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer
To condition the generation process of such a model with a user-specified sequence, a popular approach is to take that conditioning sequence as a priming sequence and ask a Transformer decoder to generate a continuation.
Music Generation
Representation Learning
Sound
Multimedia
Audio and Speech Processing
