Search Results for author:
Found 32 papers, 18 papers with code
Cost-aware Pre-training for Multiclass Cost-sensitive Deep Learning
Deep learning has been one of the most prominent machine learning techniques nowadays, being the state-of-the-art on a broad range of applications where automatic feature extraction is needed.
Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder
The vector representations of fixed dimensionality for words (in text) offered by Word2Vec have been shown to be very useful in many application scenarios, in particular due to the semantic information they carry.
Cost-Sensitive Deep Learning with Layer-Wise Cost Estimation
While deep neural networks have succeeded in several visual applications, such as object recognition, detection, and localization, by reaching very high classification accuracies, it is important to note that many real-world applications demand varying costs for different types of misclassification errors, thus requiring cost-sensitive classification algorithms.
libact: Pool-based Active Learning in Python
libact is a Python package designed to make active learning easier for general users.
Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar.
Supervised and Unsupervised Transfer Learning for Question Answering
Although transfer learning has been shown to be successful for tasks like object and speech recognition, its applicability to question answering (QA) has yet to be well-studied.
Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval
Deep neural networks have been investigated in learning latent representations of medical images, yet most of the studies limit their approach in a single supervised convolutional neural network (CNN), which usually rely heavily on a large scale annotated dataset for training.
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar.
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
Recent research has shown that word embedding spaces learned from text corpora of different languages can be aligned without any parallel data supervision.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis
We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.
Towards Unsupervised Speech-to-Text Translation
We present a framework for building speech-to-text translation (ST) systems using only monolingual speech and text corpora, in other words, speech utterances from a source language and independent text from a target language.
Unsupervised Clinical Language Translation
As patients' access to their doctors' clinical notes becomes common, translating professional, clinical jargon to layperson-understandable language is essential to improve patient-clinician communication.
An Unsupervised Autoregressive Model for Speech Representation Learning
This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations.
Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
For an input text, it is simultaneously passed into BERT and the Tacotron-2 encoder.
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.
Generative Pre-Training for Speech with Autoregressive Predictive Coding
Learning meaningful and general representations from unannotated speech that are applicable to a wide range of tasks remains challenging.
Clinical Text Summarization with Syntax-Based Negation and Semantic Concept Identification
In the era of clinical information explosion, a good strategy for clinical text summarization is helpful to improve the clinical workflow.
Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
Training objectives based on predictive coding have recently been shown to be very effective at learning meaningful representations from unlabeled speech.
Vector-Quantized Autoregressive Predictive Coding
Autoregressive Predictive Coding (APC), as a self-supervised objective, has enjoyed success in learning representations from large amounts of unlabeled data, and the learned representations are rich for many downstream tasks.
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
Besides conducting a self-supervised masked language modeling task on the two individual modules using unpaired speech and text, SPLAT aligns representations from the two modules in a shared latent space using a small amount of paired speech and text.
Similarity Analysis of Self-Supervised Speech Representations
We also design probing tasks to study the correlation between the models' pre-training loss and the amount of specific speech information contained in their learned representations.
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Self-supervised speech representations have been shown to be effective in a variety of speech applications.
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation
Audio tagging is an active research area and has a wide range of applications.
Ranked #6 on
Audio Classification
on FSD50K
(using extra training data)
AST: Audio Spectrogram Transformer
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.
Ranked #1 on
Audio Classification
on Speech Commands
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
In particular, when compared to published models such as conformer-based wav2vec~2. 0 and HuBERT, our model shows~5\% to~10\% relative WER reduction on the test-clean and test-other subsets.
Ranked #1 on
Speech Recognition
on LibriSpeech test-clean
(using extra training data)
SSAST: Self-Supervised Audio Spectrogram Transformer
However, pure Transformer models tend to require more training data compared to CNNs, and the success of the AST relies on supervised pretraining that requires a large amount of labeled data and a complex training pipeline, thus limiting the practical usage of AST.
Ranked #1 on
Spoken Command Recognition
on Speech Command v2
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training
We build a single encoder with the BERT objective on unlabeled text together with the w2v-BERT objective on unlabeled speech.
Speech-to-Speech Translation For A Real-world Unwritten Language
We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units
We enhance the model performance by subword prediction in the first-pass decoder, advanced two-pass decoder architecture design and search strategy, and better training regularization.
SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
Ranked #1 on
Machine Translation
on flores95-devtest eng-X
Automatic Speech Recognition
Speech-to-Speech Translation
+3
CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders
Large-scale self-supervised pre-trained speech encoders outperform conventional approaches in speech recognition and translation tasks.
Seamless: Multilingual Expressive and Streaming Speech Translation
In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion.
