Search Results for author:
Found 12 papers, 6 papers with code
On the Evaluation of Speech Foundation Models for Spoken Language Understanding
To answer this, we perform an extensive evaluation of multiple supervised and self-supervised SFMs using several evaluation protocols: (i) frozen SFMs with a lightweight prediction head, (ii) frozen SFMs with a complex prediction head, and (iii) fine-tuned SFMs with a lightweight prediction head.
Self-Supervised Speech Representations are More Phonetic than Semantic
Self-supervised speech models (S3Ms) have become an effective backbone for speech applications.
What Do Self-Supervised Speech Models Know About Words?
Many self-supervised speech models (S3Ms) have been introduced over the last few years, improving performance and data efficiency on various speech tasks.
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
In this work, we introduce several new annotated SLU benchmark tasks based on freely available speech data, which complement existing benchmarks and address gaps in the SLU evaluation landscape.
Comparative layer-wise analysis of self-supervised speech models
We further investigate the utility of our analyses for downstream tasks by comparing the property trends with performance on speech recognition and spoken language understanding tasks.
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning
We propose to select layers based on the variability of their hidden states given a task-specific corpus.
On the Use of External Data for Spoken Named Entity Recognition
In this work we focus on low-resource spoken named entity recognition (NER) and address the question: Beyond self-supervised pre-training, how can we use external speech and/or text data that are not annotated for the task?
SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech
Historically these have focused on automatic speech recognition (ASR), speaker identification, or other lower-level tasks.
Ranked #1 on
Named Entity Recognition (NER)
on SLUE
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
Layer-wise Analysis of a Self-supervised Speech Representation Model
Recently proposed self-supervised learning approaches have been successful for pre-training speech representation models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Taskology: Utilizing Task Relations at Scale
Many computer vision tasks address the problem of scene understanding and are naturally interrelated e. g. object classification, detection, scene segmentation, depth estimation, etc.
Improving Semantic Segmentation through Spatio-Temporal Consistency Learned from Videos
We leverage unsupervised learning of depth, egomotion, and camera intrinsics to improve the performance of single-image semantic segmentation, by enforcing 3D-geometric and temporal consistency of segmentation masks across video frames.
On the Contributions of Visual and Textual Supervision in Low-Resource Semantic Speech Retrieval
Recent work has shown that speech paired with images can be used to learn semantically meaningful speech representations even without any textual supervision.
