Search Results for author:
Found 27 papers, 13 papers with code
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
We build a single encoder with the BERT objective on unlabeled text together with the w2v-BERT objective on unlabeled speech.
SSAST: Self-Supervised Audio Spectrogram Transformer
However, pure Transformer models tend to require more training data compared to CNNs, and the success of the AST relies on supervised pretraining that requires a large amount of labeled data and a complex training pipeline, thus limiting the practical usage of AST.
Ranked #1 on
Spoken Command Recognition
on Speech Command v2
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
In particular, when compared to published models such as conformer-based wav2vec~2. 0 and HuBERT, our model shows~5\% to~10\% relative WER reduction on the test-clean and test-other subsets.
Ranked #1 on
Speech Recognition
on LibriSpeech test-clean
(using extra training data)
AST: Audio Spectrogram Transformer
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.
Ranked #2 on
Time Series
on Speech Commands
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation
Audio tagging is an active research area and has a wide range of applications.
Ranked #3 on
Audio Tagging
on AudioSet
(using extra training data)
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Self-supervised speech representations have been shown to be effective in a variety of speech applications.
Similarity Analysis of Self-Supervised Speech Representations
We also design probing tasks to study the correlation between the models' pre-training loss and the amount of specific speech information contained in their learned representations.
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
Besides conducting a self-supervised masked language modeling task on the two individual modules using unpaired speech and text, SPLAT aligns representations from the two modules in a shared latent space using a small amount of paired speech and text.
Vector-Quantized Autoregressive Predictive Coding
Autoregressive Predictive Coding (APC), as a self-supervised objective, has enjoyed success in learning representations from large amounts of unlabeled data, and the learned representations are rich for many downstream tasks.
Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
Training objectives based on predictive coding have recently been shown to be very effective at learning meaningful representations from unlabeled speech.
Clinical Text Summarization with Syntax-Based Negation and Semantic Concept Identification
In the era of clinical information explosion, a good strategy for clinical text summarization is helpful to improve the clinical workflow.
Generative Pre-Training for Speech with Autoregressive Predictive Coding
Learning meaningful and general representations from unannotated speech that are applicable to a wide range of tasks remains challenging.
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.
Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
For an input text, it is simultaneously passed into BERT and the Tacotron-2 encoder.
An Unsupervised Autoregressive Model for Speech Representation Learning
This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations.
Unsupervised Clinical Language Translation
As patients' access to their doctors' clinical notes becomes common, translating professional, clinical jargon to layperson-understandable language is essential to improve patient-clinician communication.
Towards Unsupervised Speech-to-Text Translation
We present a framework for building speech-to-text translation (ST) systems using only monolingual speech and text corpora, in other words, speech utterances from a source language and independent text from a target language.
Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis
We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
Recent research has shown that word embedding spaces learned from text corpora of different languages can be aligned without any parallel data supervision.
Automatic Speech Recognition
Cross-Lingual Word Embeddings
+3
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar.
Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval
Deep neural networks have been investigated in learning latent representations of medical images, yet most of the studies limit their approach in a single supervised convolutional neural network (CNN), which usually rely heavily on a large scale annotated dataset for training.
Supervised and Unsupervised Transfer Learning for Question Answering
Although transfer learning has been shown to be successful for tasks like object and speech recognition, its applicability to question answering (QA) has yet to be well-studied.
Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar.
libact: Pool-based Active Learning in Python
libact is a Python package designed to make active learning easier for general users.
Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation
While deep neural networks have succeeded in several visual applications, such as object recognition, detection, and localization, by reaching very high classification accuracies, it is important to note that many real-world applications demand varying costs for different types of misclassification errors, thus requiring cost-sensitive classification algorithms.
Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
The vector representations of fixed dimensionality for words (in text) offered by Word2Vec have been shown to be very useful in many application scenarios, in particular due to the semantic information they carry.
Cost-aware Pre-training for Multiclass Cost-sensitive Deep Learning
Deep learning has been one of the most prominent machine learning techniques nowadays, being the state-of-the-art on a broad range of applications where automatic feature extraction is needed.
