Search Results for author:
Found 8 papers, 4 papers with code
Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
We present Mockingjay as a new speech representation learning approach, where bidirectional Transformer encoders are pre-trained on a large amount of unlabeled speech.
Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation
We use the representations with two downstream tasks, speaker identification, and phoneme classification.
SUPERB: Speech processing Universal PERformance Benchmark
SUPERB is a leaderboard to benchmark the performance of a shared model across a wide range of speech processing tasks with minimal architecture changes and labeled data.
SpeechNet: A Universal Modularized Model for Speech Processing Tasks
The multi-task learning of a wide variety of speech processing tasks with a universal model has not been studied.
BERT's output layer recognizes all hidden layers? Some Intriguing Phenomena and a simple way to boost BERT
Although Bidirectional Encoder Representations from Transformers (BERT) have achieved tremendous success in many natural language processing (NLP) tasks, it remains a black box.
Input-independent Attention Weights Are Expressive Enough: A Study of Attention in Self-supervised Audio Transformers
In this paper, we seek solutions for reducing the computation complexity of transformer-based models for speech representation learning.
Leveraging Sequence Embedding and Convolutional Neural Network for Protein Function Prediction
In contrast, most of the existing methods delete the rare protein functions to reduce the label space.
A Large-Scale Evaluation of Speech Foundation Models
In this work, we establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the paradigm for speech.
