Search Results for author:
Found 17 papers, 6 papers with code
Cosmos World Foundation Model Platform for Physical AI
In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups.
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
We present Dynamic-SUPERB Phase-2, an open and evolving benchmark for the comprehensive evaluation of instruction-based universal speech models.
Measuring Sound Symbolism in Audio-visual Models
Audio-visual pre-trained models have gained substantial attention recently and demonstrated superior performance on various audio-visual tasks.
SpeechPrompt: Prompting Speech Language Models for Speech Processing Tasks
Motivated by the strengths of prompting, we are the first to explore the potential of prompting speech LMs in the domain of speech processing.
A Large-Scale Evaluation of Speech Foundation Models
In this work, we establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the paradigm for speech.
VMCML: Video and Music Matching via Cross-Modality Lifting
We propose a content-based system for matching video and background music.
SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks
For speech processing, SpeechPrompt shows its high parameter efficiency and competitive performance on a few speech classification tasks.
Ranked #17 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Ensemble knowledge distillation of self-supervised speech models
We tried two different aggregation techniques, layerwise-average and layerwise-concatenation, to the representations of different teacher models and found that the former was more effective.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores
Mean opinion score (MOS) is a typical subjective evaluation metric for speech synthesis systems.
SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks
We report in this paper the first exploration of the prompt tuning paradigm for speech processing tasks based on Generative Spoken Language Model (GSLM).
CLA-NeRF: Category-Level Articulated Neural Radiance Field
We propose CLA-NeRF -- a Category-Level Articulated Neural Radiance Field that can perform view synthesis, part segmentation, and articulated pose estimation.
Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios.
Leveraging Sequence Embedding and Convolutional Neural Network for Protein Function Prediction
In contrast, most of the existing methods delete the rare protein functions to reduce the label space.
Membership Inference Attacks Against Self-supervised Speech Models
Recently, adapting the idea of self-supervised learning (SSL) on continuous speech has started gaining attention.
SUPERB: Speech processing Universal PERformance Benchmark
SUPERB is a leaderboard to benchmark the performance of a shared model across a wide range of speech processing tasks with minimal architecture changes and labeled data.
Utilizing Self-supervised Representations for MOS Prediction
In this paper, we use self-supervised pre-trained models for MOS prediction.
Toward Robust Long Range Policy Transfer
We also design two regularization terms to improve the diversity and utilization rate of the primitives in the pre-training phase.
