Search Results for author:
Found 59 papers, 26 papers with code
Automatic Long Audio Alignment and Confidence Scoring for Conversational Arabic Speech
In a second pass, a more restricted LM is generated for each audio segment, and unsupervised acoustic model adaptation is applied.
Development of a TV Broadcasts Speech Recognition System for Qatari Arabic
A major problem with dialectal Arabic speech recognition is due to the sparsity of speech resources.
Deep learning for monaural speech separation
In this paper, we study deep learning for monaural speech separation.
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation
In this paper, we explore joint optimization of masking functions and deep recurrent neural networks for monaural source separation tasks, including monaural speech separation, monaural singing voice separation, and speech denoising.
Semantic Image Inpainting with Deep Generative Models
In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data.
Landmark-based consonant voicing detection on multilingual corpora
Three consonant voicing classifiers were developed: (1) manually selected acoustic features anchored at a phonetic landmark, (2) MFCCs (either averaged across the segment or anchored at the landmark), and(3) acoustic features computed using a convolutional neural network (CNN).
Clustering-based Phonetic Projection in Mismatched Crowdsourcing Channels for Low-resourced ASR
We evaluate our techniques using mismatched transcriptions for Cantonese speech acquired from native English and Mandarin speakers.
Performance Improvements of Probabilistic Transcript-adapted ASR with Recurrent Neural Network and Language-specific Constraints
Mismatched transcriptions have been proposed as a mean to acquire probabilistic transcriptions from non-native speakers of a language. Prior work has demonstrated the value of these transcriptions by successfully adapting cross-lingual ASR systems for different tar-get languages.
Dilated Recurrent Neural Networks
To provide a theory-based quantification of the architecture's advantages, we introduce a memory capacity measure, the mean recurrent length, which is more suitable for RNNs with long skip connections than existing measures.
Ranked #24 on
Sequential Image Classification
on Sequential MNIST
Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition
The performance of automatic speech recognition systems degrades with increasing mismatch between the training and testing scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Linguistic unit discovery from multi-modal inputs in unwritten languages: Summary of the "Speaking Rosetta" JSALT 2017 Workshop
We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding the discovery of linguistic units (subwords and words) in a language without orthography.
Deep Learning Based Speech Beamforming
On the other hand, deep learning based enhancement approaches are able to learn complicated speech distributions and perform efficient inference, but they are unable to deal with variable number of input channels.
Bayesian Models for Unit Discovery on a Very Low Resource Language
Developing speech technologies for low-resource languages has become a very active research field over the last decade.
Improved ASR for Under-Resourced Languages Through Multi-Task Learning with Acoustic Landmarks
Furui first demonstrated that the identity of both consonant and vowel can be perceived from the C-V transition; later, Stevens proposed that acoustic landmarks are the primary cues for speech perception, and that steady-state regions are secondary or supplemental.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
When CTC Training Meets Acoustic Landmarks
In this paper, the convergence properties of CTC are improved by incorporating acoustic landmarks.
AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
On the other hand, CVAE training is simple but does not come with the distribution-matching property of a GAN.
Fast transcription of speech in low-resource languages
We present software that, in only a few hours, transcribes forty hours of recorded speech in a surprise language, using only a few tens of megabytes of noisy text in that language, and a zero-resource grapheme to phoneme (G2P) table.
Continuous Convolutional Neural Network forNonuniform Time Series
Convolutional neural network (CNN) for time series data implicitly assumes that the data are uniformly sampled, whereas many event-based and multi-modal data are nonuniform or have heterogeneous sampling rates.
F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder
Recently, AutoVC, a conditional autoencoders (CAEs) based method achieved state-of-the-art results by disentangling the speaker identity and speech content using information-constraining bottlenecks, and it achieves zero-shot conversion by swapping in a different speaker's identity embedding to synthesize a new voice.
Unsupervised Speech Decomposition via Triple Information Bottleneck
Speech information can be roughly decomposed into four components: language content, timbre, pitch, and rhythm.
Automatic Estimation of Intelligibility Measure for Consonants in Speech
In this article, we provide a model to estimate a real-valued measure of the intelligibility of individual speech segments.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages
Only a handful of the world's languages are abundant with the resources that enable practical applications of speech processing technologies.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Identify Speakers in Cocktail Parties with End-to-End Attention
In scenarios where multiple speakers talk at the same time, it is important to be able to identify the talkers accurately.
Evaluating Automatically Generated Phoneme Captions for Images
For this, first an Image2Speech system was implemented which generates image captions consisting of phoneme sequences.
Deep F-measure Maximization for End-to-End Speech Understanding
We propose a differentiable approximation to the F-measure and train the network with this objective using standard backpropagation.
How Phonotactics Affect Multilingual and Zero-shot ASR Performance
Furthermore, we find that a multilingual LM hurts a multilingual ASR system's performance, and retaining only the target language's phonotactic data in LM training is preferable.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Show and Speak: Directly Synthesize Spoken Description of Images
This paper proposes a new model, referred to as the show and speak (SAS) model that, for the first time, is able to directly synthesize spoken descriptions of images, bypassing the need for any text or phonemes.
Interpretable Visual Reasoning via Induced Symbolic Space
Finally, we achieve a higher level of interpretability by imposing OCCAM on the objects represented in the induced symbolic concept space.
Ranked #3 on
Visual Question Answering (VQA)
on CLEVR
Multi-Decoder DPRNN: High Accuracy Source Counting and Separation
Beyond the model, we also propose a metric on how to evaluate source separation with variable number of speakers.
Ranked #5 on
Speech Separation
on WSJ0-4mix
Seeing is Knowing! Fact-based Visual Question Answering using Knowledge Graph Embeddings
Therefore, being able to reason over incomplete KGs for QA is a critical requirement in real-world applications that has not been addressed extensively in the literature.
Worldly Wise (WoW) - Cross-Lingual Knowledge Fusion for Fact-based Visual Spoken-Question Answering
Three sub-tasks are proposed: (1) speech-to-text based, (2) end-to-end, without speech-to-text as an intermediate component, and (3) cross-lingual, in which the question is spoken in a language different from that in which the KG is recorded.
Global Rhythm Style Transfer Without Text Transcriptions
In this paper, we propose AutoPST, which can disentangle global prosody style from speech without relying on any text transcriptions.
Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifold
This paper defines fair principal component analysis (PCA) as minimizing the maximum mean discrepancy (MMD) between dimensionality-reduced conditional distributions of different protected classes.
Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition
In this paper, we 1) investigate the influence of different factors (i. e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language; 2) provide an analysis of which phones transfer well across languages and which do not in order to understand the limitations of and areas for further improvement for automatic phone inventory creation; and 3) present different methods to build a phone inventory of an unseen language in an unsupervised way.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottlenecks
SpeechSplit can perform aspect-specific voice conversion by disentangling speech into content, rhythm, pitch, and timbre using multiple autoencoders in an unsupervised manner.
Visualizations of Complex Sequences of Family-Infant Vocalizations Using Bag-of-Audio-Words Approach Based on Wav2vec 2.0 Features
We demonstrate that our high-quality visualizations capture major types of family vocalization interactions, in categories indicative of mental, behavioral, and developmental health, for both labeled and unlabeled LB audio.
WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Models
We show that WavPrompt is a few-shot learner that can perform speech understanding tasks better than a naive text baseline.
Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition
An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untranscribed speech waveforms in that language; 2) a collection of texts written in that language without access to any transcribed speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Equivariance Discovery by Learned Parameter-Sharing
Designing equivariance as an inductive bias into deep-nets has been a prominent approach to build effective models, e. g., a convolutional neural network incorporates translation equivariance.
ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to downstream tasks.
End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions
Zero-shot voice conversion is becoming an increasingly popular research topic, as it promises the ability to transform speech to sound like any speaker.
Forget-free Continual Learning with Winning Subnetworks
Inspired by Lottery Ticket Hypothesis that competitive subnetworks exist within a dense network, we propose a continual learning method referred to as Winning SubNetworks (WSN), which sequentially learns and selects an optimal subnetwork for each task.
Dual-Path Cross-Modal Attention for better Audio-Visual Speech Extraction
Most current research upsamples the visual features along the time dimension so that audio and video features are able to align in time.
SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation
To the best of our knowledge, this is the first attempt to apply mixup in NLP while preserving the meaning of a specific word.
Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audio
To perform automatic family audio analysis, past studies have collected recordings using phone, video, or audio-only recording devices like LENA, investigated supervised learning methods, and used or fine-tuned general-purpose embeddings learned from large pretrained models.
INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition
INTapt is trained simultaneously in the following two manners: (1) adversarial training to reduce accent feature dependence between the original input and the prompt-concatenated input and (2) training to minimize CTC loss for improving ASR performance to a prompt-concatenated input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Theory of Unsupervised Speech Recognition
Unsupervised speech recognition (ASR-U) is the problem of learning automatic speech recognition (ASR) systems from unpaired speech-only and text-only corpora.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P) Transduction
Text-to-Text Transfer Transformer (T5) has recently been considered for the Grapheme-to-Phoneme (G2P) transduction.
Enhancing Child Vocalization Classification in Multi-Channel Child-Adult Conversations Through Wav2vec2 Children ASR Features
In this study, we leverage the self-supervised learning model, Wav2Vec 2. 0 (W2V2), pretrained on 4300h of home recordings of children under 5 years old, to build a unified system that performs both speaker diarization (SD) and vocalization classification (VC) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matching
Training unsupervised speech recognition systems presents challenges due to GAN-associated instability, misalignment between speech and text, and significant memory demands.
HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models
We further extend our method to a novel image editing task: substituting the subject in an image through textual manipulations.
Analysis of Self-Supervised Speech Models on Children's Speech and Infant Vocalizations
To understand why self-supervised learning (SSL) models have empirically achieved strong performances on several speech-processing downstream tasks, numerous studies have focused on analyzing the encoded information of the SSL layer representations in adult speech.
AdaMER-CTC: Connectionist Temporal Classification with Adaptive Maximum Entropy Regularization for Automatic Speech Recognition
In Automatic Speech Recognition (ASR) systems, a recurring obstacle is the generation of narrowly focused output distributions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion
Through a series of observations, we find that the prompt choice significantly affects the calibration in CLIP, where the prompts leading to higher text feature dispersion result in better-calibrated predictions.
Syn2Vec: Synset Colexification Graphs for Lexical Semantic Similarity
In this paper we focus on patterns of colexification (co-expressions of form-meaning mapping in the lexicon) as an aspect of lexical-semantic organization, and use them to build large scale synset graphs across BabelNet’s typologically diverse set of 499 world languages.
Self-supervised Semantic-driven Phoneme Discovery for Zero-resource Speech Recognition
Phonemes are defined by their relationship to words: changing a phoneme changes the word.
Context-Aware Automatic Text Simplification of Health Materials in Low-Resource Domains
We also show the enhanced accuracy of our system over directly-supervised neural methods in this low-resource setting.
