Search Results for author:
Found 16 papers, 14 papers with code
CiwaGAN: Articulatory information exchange
This paper introduces CiwaGAN, a model of human spoken language acquisition that combines unsupervised articulatory modeling with an unsupervised model of information exchange through the auditory modality.
Deep Speech Synthesis from MRI-Based Articulatory Representations
Finally, through a series of ablations, we show that the proposed MRI representation is more comprehensive than EMA and identify the most suitable MRI feature subset for articulatory synthesis.
Speaker-Independent Acoustic-to-Articulatory Speech Inversion
To build speech processing methods that can handle speech as naturally as humans, researchers have explored multiple ways of building an invertible mapping from speech to an interpretable space.
A Fast and Accurate Pitch Estimation Algorithm Based on the Pseudo Wigner-Ville Distribution
Estimation of fundamental frequency (F0) in voiced segments of speech signals, also known as pitch tracking, plays a crucial role in pitch synchronous speech analysis, speech synthesis, and speech manipulation.
Articulation GAN: Unsupervised modeling of articulatory learning
Articulatory analysis suggests that the network learns to control articulators in a similar manner to humans during speech production.
Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech
Recent self-supervised learning (SSL) models have proven to learn rich representations of speech, which can readily be utilized by diverse downstream tasks.
Deep Speech Synthesis from Articulatory Representations
In the articulatory synthesis task, speech is synthesized from input features containing information about the physical behavior of the human vocal tract.
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
Our paper takes a step towards real-world physical commonsense reasoning by contributing PACS: the first audiovisual benchmark annotated for physical commonsense attributes.
Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity
We demonstrate the effectiveness of our approach in language family classification, speech recognition, and speech synthesis tasks.
ESPnet2-TTS: Extending the Edge of TTS Research
This paper describes ESPnet2-TTS, an end-to-end text-to-speech (E2E-TTS) toolkit.
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
In order to accelerate progress towards understudied modalities and tasks while ensuring real-world robustness, we release MultiBench, a systematic and unified large-scale benchmark spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas.
Understanding the Tradeoffs in Client-side Privacy for Downstream Speech Tasks
As users increasingly rely on cloud-based computing services, it is important to ensure that uploaded speech data remains private.
Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
In this work, we propose algorithms for cross-modal generalization: a learning paradigm to train a model that can (1) quickly perform new tasks in a target modality (i. e. meta-learning) and (2) doing so while being trained on a different source modality.
Automatically Identifying Language Family from Acoustic Examples in Low Resource Scenarios
Existing multilingual speech NLP works focus on a relatively small subset of languages, and thus current linguistic understanding of languages predominantly stems from classical approaches.
LEAF: A Benchmark for Federated Settings
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.
Machine Learning for Exam Triage
In this project, we extend the state-of-the-art CheXNet (Rajpurkar et al. [2017]) by making use of the additional non-image features in the dataset.
