Search Results for author:
Found 153 papers, 66 papers with code
Speak: A Toolkit Using Amazon Mechanical Turk to Collect and Validate Speech Audio Recordings
We present Speak, a toolkit that allows researchers to crowdsource speech audio recordings using Amazon Mechanical Turk (MTurk).
Curiosity-driven Red-teaming for Large Language Models
To probe when an LLM generates unwanted content, the current paradigm is to recruit a \textit{red team} of human testers to design input prompts (i. e., test cases) that elicit undesirable responses from LLMs.
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective
We use linear probes to estimate the mutual information between the target information and learned representations, showing another insight into the accessibility to the target information from speech representations.
R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces
This paper introduces Robust Spin (R-Spin), a data-efficient domain-specific self-supervision method for speaker and noise-invariant speech representations by learning discrete acoustic units with speaker-invariant clustering (Spin).
Audio-Visual Neural Syntax Acquisition
We study phrase structure induction from visually-grounded speech.
Self-Specialization: Uncovering Latent Expertise within Large Language Models
Recent works have demonstrated the effectiveness of self-alignment in which a large language model is, by itself, aligned to follow general instructions through the automatic generation of instructional data using a handful of human-written seeds.
Joint Audio and Speech Understanding
Humans are surrounded by audio signals that include both speech and non-speech sounds.
Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning
How can we perform computations over natural language representations to solve tasks that require symbolic and numeric reasoning?
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Despite their impressive capabilities, large language models (LLMs) are prone to hallucinations, i. e., generating content that deviates from facts seen during pretraining.
Improved Cross-Lingual Transfer Learning For Automatic Speech Translation
Having a single model that supports multiple translation tasks is desirable.
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering
We propose EAR, a query Expansion And Reranking approach for improving passage retrieval, with the application to open-domain question answering.
Entailment as Robust Self-Learner
Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.
Multi-class Classification
Natural Language Understanding
+1
SAIL: Search-Augmented Instruction Learning
Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information.
Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages
Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each.
Listen, Think, and Understand
On the other hand, modern large language models (LLMs) exhibit emerging reasoning ability but they lack audio perception capabilities.
Ranked #3 on
Music Question Answering
on MusicQA
(using extra training data)
Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Self-supervised speech representation models have succeeded in various tasks, but improving them for content-related problems using unlabeled data is challenging.
Interpretable Unified Language Checking
Despite recent concerns about undesirable behaviors generated by large language models (LLMs), including non-factual, biased, and hateful language, we find LLMs are inherent multi-task language checkers based on their latent representations of natural and social knowledge.
What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructions
Spatio-temporal grounding describes the task of localizing events in space and time, e. g., in video data, based on verbal descriptions only.
Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning
Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora.
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
In this work, we explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics: stress tests with synthetic data.
On Unsupervised Uncertainty-Driven Speech Pseudo-Label Filtering and Model Calibration
Dropout-based Uncertainty-driven Self-Training (DUST) proceeds by first training a teacher model on source domain labeled data.
PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation
Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes.
C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval
Inspired by the fact that English text-video retrieval outperforms other languages, we train a student model using input text in different languages to match the cross-modal predictions from teacher models using input text in English.
Contrastive Audio-Visual Masked Autoencoder
In this paper, we first extend the recent Masked Auto-Encoder (MAE) model from a single modality to audio-visual multi-modalities.
Ranked #1 on
Audio Tagging
on AudioSet
(using extra training data)
UAVM: Towards Unifying Audio and Visual Models
Conventional audio-visual models have independent audio and video branches.
Ranked #2 on
Multi-modal Classification
on AudioSet
(using extra training data)
Developing a Series of AI Challenges for the United States Department of the Air Force
Through a series of federal initiatives and orders, the U. S. Government has been making a concerted effort to ensure American leadership in AI.
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation
We combine state-of-the-art multilingual acoustic frame-level speech representation learning model XLS-R with the Language Agnostic BERT Sentence Embedding (LaBSE) model to create an utterance-level multimodal multilingual speech encoder SAMU-XLSR.
Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition
Recognizing human non-speech vocalizations is an important task and has broad applications such as automatic sound transcription and health condition monitoring.
Ranked #1 on
Audio Classification
on VocalSound
Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment
Automatic pronunciation assessment is an important technology to help self-directed language learners.
Ranked #2 on
Phone-level pronunciation scoring
on speechocean762
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence embeddings.
Ranked #13 on
Semantic Textual Similarity
on STS16
Simple and Effective Unsupervised Speech Synthesis
We introduce the first unsupervised speech synthesis system based on a simple, yet effective recipe.
CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification
Audio classification is an active research area with a wide range of applications.
Controlling the Focus of Pretrained Language Generation Models
This work aims to develop a control mechanism by which a user can select spans of context as "highlights" for the model to focus on, and generate relevant output.
Everything at Once - Multi-Modal Fusion Transformer for Video Retrieval
In this work, we present a multi-modal, modality agnostic fusion transformer that learns to exchange information between multiple modalities, such as video, audio, and text, and integrate them into a fused representation in a joined multi-modal embedding space.
Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval
Multi-modal learning from video data has seen increased attention recently as it allows to train semantically meaningful embeddings without human annotation enabling tasks like zero-shot retrieval and classification.
Routing with Self-Attention for Multimodal Capsule Networks
We present a new multimodal capsule network that allows us to leverage the strength of capsules in the context of a multimodal learning framework on large amounts of video data.
Cascaded Multilingual Audio-Visual Learning from Videos
In this paper, we explore self-supervised audio-visual models that learn from instructional videos.
SSAST: Self-Supervised Audio Spectrogram Transformer
However, pure Transformer models tend to require more training data compared to CNNs, and the success of the AST relies on supervised pretraining that requires a large amount of labeled data and a complex training pipeline, thus limiting the practical usage of AST.
Ranked #1 on
Spoken Command Recognition
on Speech Command v2
Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
These results show that models trained on other datasets and then evaluated on Spoken ObjectNet tend to perform poorly due to biases in other datasets that the models have learned.
Magic dust for cross-lingual adaptation of monolingual wav2vec-2.0
We propose a simple and effective cross-lingual transfer learning method to adapt monolingual wav2vec-2. 0 models for Automatic Speech Recognition (ASR) in resource-scarce languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
Are end-to-end text-to-speech (TTS) models over-parametrized?
An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models
Prompt-based knowledge probing for 1-hop relations has been used to measure how much world knowledge is stored in pretrained language models.
Interpretable Propaganda Detection in News Articles
Online users today are exposed to misleading and propagandistic news articles and media posts on a daily basis.
Mitigating Biases in Toxic Language Detection through Invariant Rationalization
Automatic detection of toxic language plays an essential role in protecting social media users, especially minority groups, from verbal abuse.
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
We investigate the existence of sparse subnetworks in pre-trained speech SSL models that achieve even better low-resource ASR results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Cross-Modal Discrete Representation Learning
Recent advances in representation learning have demonstrated an ability to represent information from different modalities such as video, text, and audio in a single high-level embedding vector.
Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
With this in mind, the descriptions people generate for videos of different dynamic events can greatly improve our understanding of the key information of interest in each video.
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos
Multimodal self-supervised learning is getting more and more attention as it allows not only to train large networks without human supervision but also to search and retrieve data across various modalities.
AST: Audio Spectrogram Transformer
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.
Ranked #1 on
Audio Classification
on Speech Commands
Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
Cooperative Self-training of Machine Reading Comprehension
Pretrained language models have significantly improved the performance of downstream language understanding tasks, including extractive question answering, by providing high-quality contextualized word embeddings.
Ranked #1 on
Question Answering
on MRQA out-of-domain
Extractive Question-Answering
Machine Reading Comprehension
+6
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation
Audio tagging is an active research area and has a wide range of applications.
Ranked #6 on
Audio Classification
on FSD50K
(using extra training data)
Knowledge Grounded Conversational Symptom Detection with Graph Memory Networks
Given a set of explicit symptoms provided by the patient to initiate a dialog for diagnosing, the system is trained to collect implicit symptoms by asking questions, in order to collect more information for making an accurate diagnosis.
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision.
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Self-supervised speech representations have been shown to be effective in a variety of speech applications.
Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretraining
Much recent work on Spoken Language Understanding (SLU) is limited in at least one of three ways: models were trained on oracle text input and neglected ASR errors, models were trained to predict only intents without the slot values, or models were trained on a large amount of in-house data.
Similarity Analysis of Self-Supervised Speech Representations
We also design probing tasks to study the correlation between the models' pre-training loss and the amount of specific speech information contained in their learned representations.
We Can Detect Your Bias: Predicting the Political Ideology of News Articles
We explore the task of predicting the leading political ideology or bias of news articles.
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
On the other hand, we find that the set of sampling algorithms that satisfies these properties performs on par with the existing sampling algorithms.
AutoKG: Constructing Virtual Knowledge Graphs from Unstructured Documents for Question Answering
Knowledge graphs (KGs) have the advantage of providing fine-grained detail for question-answering systems.
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos
Further, we propose a tri-modal model that jointly processes raw audio, video, and text captions from videos to learn a multi-modal semantic embedding space useful for text-video retrieval.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
The audio encoder is trained to perform a speech-translation retrieval task in a contrastive learning framework.
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Probabilistic Latent Variable Models (LVMs) provide an alternative to self-supervised learning approaches for linguistic representation learning from speech.
Prototypical Q Networks for Automatic Conversational Diagnosis and Few-Shot New Disease Adaption
Experiments showed that the ProtoQN significantly outperformed the baseline DQN model in both supervised and few-shot learning scenarios, and achieves state-of-the-art few-shot learning performances.
Vector-Quantized Autoregressive Predictive Coding
Autoregressive Predictive Coding (APC), as a self-supervised objective, has enjoyed success in learning representations from large amounts of unlabeled data, and the learned representations are rich for many downstream tasks.
What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context
Alternatively, we can profile entire news outlets and look for those that are likely to publish fake or biased content.
Similarity Analysis of Contextual Word Representation Models
We use existing and novel similarity measures that aim to gauge the level of localization of information in the deep models, and facilitate the investigation of which design factors affect model similarity, without requiring any external linguistic annotation.
Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
Training objectives based on predictive coding have recently been shown to be very effective at learning meaningful representations from unlabeled speech.
SemEval-2016 Task 3: Community Question Answering
This paper describes the SemEval--2016 Task 3 on Community Question Answering, which we offered in English and Arabic.
SemEval-2015 Task 3: Answer Selection in Community Question Answering
Community Question Answering (cQA) provides new interesting research directions to the traditional Question Answering (QA) field, e. g., the exploitation of the interaction between users and the structure of related posts.
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
What differentiates this paper from prior work on speech unit learning is the choice of training objective.
On the Linguistic Representational Power of Neural Machine Translation Models
(iii) Do the representations capture lexical semantics?
Neural Multi-Task Learning for Stance Prediction
We present a multi-task learning model that leverages large amount of textual information from existing datasets to improve stance prediction.
Generative Pre-Training for Speech with Autoregressive Predictive Coding
Learning meaningful and general representations from unannotated speech that are applicable to a wide range of tasks remains challenging.
Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
Tanbih: Get To Know What You Are Reading
We introduce Tanbih, a news aggregator with intelligent analysis tools to help readers understanding what's behind a news story.
Contrastive Language Adaptation for Cross-Lingual Stance Detection
In particular, we introduce a novel contrastive language adaptation approach applied to memory networks, which ensures accurate alignment of stances in the source and target languages, and can effectively deal with the challenge of limited labeled data in the target language.
DARTS: Dialectal Arabic Transcription System
We analyze the following; transfer learning from high resource broadcast domain to low-resource dialectal domain and semi-supervised learning where we use in-domain unlabeled audio data collected from YouTube.
Automatic Fact-Checking Using Context and Discourse Information
We study the problem of automatic fact-checking, paying special attention to the impact of contextual and discourse information.
Transfer Learning from Audio-Visual Grounding to Speech Recognition
Transfer learning aims to reduce the amount of data required to excel at a new task by re-using the knowledge acquired from learning other related tasks.
Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition
End-to-end neural network systems for automatic speech recognition (ASR) are trained from acoustic features to text transcriptions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models
For an input text, it is simultaneously passed into BERT and the Tacotron-2 encoder.
FAKTA: An Automatic End-to-End Fact Checking System
We present FAKTA which is a unified framework that integrates various components of a fact checking process: document retrieval from media sources with various types of reliability, stance detection of documents with respect to given claims, evidence extraction, and linguistic analysis.
Improving Neural Language Models by Segmenting, Attending, and Predicting the Future
In this work, we propose a method that improves language modeling by learning to align the given context and the following phrase.
Ranked #24 on
Language Modelling
on WikiText-103
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?
Exposure bias has been regarded as a central problem for auto-regressive language models (LM).
Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification
There are a number of studies about extraction of bottleneck (BN) features from deep neural networks (DNNs)trained to discriminate speakers, pass-phrases and triphone states for improving the performance of text-dependent speaker verification (TD-SV).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Language Modeling with Graph Temporal Convolutional Networks
Recently, there have been some attempts to use non-recurrent neural models for language modeling.
VoiceID Loss: Speech Enhancement for Speaker Verification
In this paper, we propose VoiceID loss, a novel loss function for training a speech enhancement model to improve the robustness of speaker verification.
Team QCRI-MIT at SemEval-2019 Task 4: Propaganda Analysis Meets Hyperpartisan News Detection
In this paper, we describe our submission to SemEval-2019 Task 4 on Hyperpartisan News Detection.
An Unsupervised Autoregressive Model for Speech Representation Learning
This paper proposes a novel unsupervised autoregressive neural model for learning generic speech representations.
Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media
In the context of fake news, bias, and propaganda, we study two important but relatively under-explored problems: (i) trustworthiness estimation (on a 3-point scale) and (ii) political ideology detection (left/right bias on a 7-point scale) of entire news outlets, as opposed to evaluating individual articles.
Negative Training for Neural Dialogue Response Generation
Although deep learning models have brought tremendous advancements to the field of open-domain dialogue response generation, recent research results have revealed that the trained models have undesirable generation behaviors, such as malicious responses and generic (boring) responses.
Towards Visually Grounded Sub-Word Speech Unit Discovery
In this paper, we investigate the manner in which interpretable sub-word speech units emerge within a convolutional neural network model trained to associate raw speech waveforms with semantically related natural image scenes.
Adversarial Domain Adaptation for Stance Detection
This paper studies the problem of stance detection which aims to predict the perspective (or stance) of a given document with respect to a given claim.
Analysis Methods in Neural Language Processing: A Survey
The field of natural language processing has seen impressive progress in recent years, with neural network models replacing many of the traditional systems.
What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models
We further present a comprehensive analysis of neurons with the aim to address the following questions: i) how localized or distributed are different linguistic properties in the models?
NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks
We present a toolkit to facilitate the interpretation and understanding of neural network models.
Domain Attentive Fusion for End-to-end Dialect Identification with Unknown Target Domain
An important issue for end-to-end systems is to have some knowledge of the application domain, because the system can be vulnerable to use cases that were not seen in the training phase; such a scenario is often referred to as a domain mismatched condition.
Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
In this paper, we present a multi-modal online person verification system using both speech and visual signals.
Towards Unsupervised Speech-to-Text Translation
We present a framework for building speech-to-text translation (ST) systems using only monolingual speech and text corpora, in other words, speech utterances from a source language and independent text from a target language.
Identifying and Controlling Important Neurons in Neural Machine Translation
Neural machine translation (NMT) models learn representations containing substantial linguistic information.
On The Inductive Bias of Words in Acoustics-to-Word Models
In addition, we study three types of inductive bias, leveraging a pronunciation dictionary, word boundary annotations, and constraints on word durations.
Predicting Factuality of Reporting and Bias of News Media Sources
We present a study on predicting the factuality of reporting and bias of news media.
Frame-level speaker embeddings for text-independent speaker recognition and analysis of end-to-end model
In this paper, we propose a Convolutional Neural Network (CNN) based speaker recognition model for extracting robust speaker embeddings.
Unsupervised Representation Learning of Speech for Dialect Identification
In this paper, we explore the use of a factorized hierarchical variational autoencoder (FHVAE) model to learn an unsupervised latent representation for dialect identification (DID).
Detecting egregious responses in neural sequence-to-sequence models
We adopt an empirical methodology, in which we first create lists of egregious output sequences, and then design a discrete optimization algorithm to find input sequences that will cause the model to generate them.
Language Identification and Morphosyntactic Tagging: The Second VarDial Evaluation Campaign
We present the results and the findings of the Second VarDial Evaluation Campaign on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects.
MCE 2018: The 1st Multi-target Speaker Detection and Identification Challenge Evaluation (MCE) Plan, Dataset and Baseline System
The Multitarget Challenge aims to assess how well current speech technology is able to determine whether or not a recorded utterance was spoken by one of a large number of 'blacklisted' speakers.
Audio and Speech Processing Sound
On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition
In this paper, we study recurrent networks' ability to learn long-term dependency in the context of speech recognition.
Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition
However, it is relatively inexpensive to collect large amounts of unlabeled data from domains that we want the models to generalize to.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Study of Enhancement, Augmentation, and Autoencoder Methods for Domain Adaptation in Distant Speech Recognition
Speech recognizers trained on close-talking speech do not generalize to distant speech and the word error rate degradation can be as large as 40% absolute.
Role-specific Language Models for Processing Recorded Neuropsychological Exams
Neuropsychological examinations are an important screening tool for the presence of cognitive conditions (e. g. Alzheimer{'}s, Parkinson{'}s Disease), and require a trained tester to conduct the exam through spoken interactions with the subject.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data
In this paper, we present a partitioned variational autoencoder (PVAE) and several training objectives to learn disentangled representations, which encode not only the shared factors, but also modality-dependent ones, into separate latent variables.
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces
Recent research has shown that word embedding spaces learned from text corpora of different languages can be aligned without any parallel data supervision.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference
We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena.
Integrating Stance Detection and Fact Checking in a Unified Corpus
A reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e. g., news websites, social media, etc.
Automatic Stance Detection Using End-to-End Memory Networks
We present a novel end-to-end memory network for stance detection, which jointly (i) predicts whether a document agrees, disagrees, discusses or is unrelated with respect to a given target claim, and also (ii) extracts snippets of evidence for that prediction.
Ranked #6 on
Fake News Detection
on FNC-1
Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech
In this paper, we explore the learning of neural network embeddings for natural images and speech waveforms describing the content of those images.
Scalable Factorized Hierarchical Variational Autoencoder Training
Deep generative models have achieved great success in unsupervised learning with the ability to capture complex nonlinear relationships between latent generating factors and observations.
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input
In this paper, we explore neural network models that learn to associate segments of spoken audio captions with the semantically relevant portions of natural images that they refer to.
Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar.
Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition
Although the Siamese network with language embeddings did not achieve as good a result as the end-to-end DID system, the two approaches had good synergy when combined together in a fused system.
Sound Audio and Speech Processing
Fact Checking in Community Forums
Community Question Answering (cQA) forums are very popular nowadays, as they represent effective means for communities around particular topics to share information.
Extracting Domain Invariant Features by Unsupervised Learning for Robust Automatic Speech Recognition
The performance of automatic speech recognition (ASR) systems can be significantly compromised by previously unseen conditions, which is typically due to a mismatch between training and testing distributions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks
In this paper, we investigate the representations learned at different layers of NMT encoders.
Learning Modality-Invariant Representations for Speech and Images
In this paper, we explore the unsupervised learning of a semantic embedding space for co-occurring sensory inputs.
Supervised and Unsupervised Transfer Learning for Question Answering
Although transfer learning has been shown to be successful for tasks like object and speech recognition, its applicability to question answering (QA) has yet to be well-studied.
Learning Word Embeddings from Speech
In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar.
Spoken Language Biomarkers for Detecting Cognitive Impairment
In this study we developed an automated system that evaluates speech and language features from audio recordings of neuropsychological examinations of 92 subjects in the Framingham Heart Study.
Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems
In this work, we analyze the speech representations learned by a deep end-to-end model that is based on convolutional and recurrent layers, and trained with a connectionist temporal classification (CTC) loss.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
MIT-QCRI Arabic Dialect Identification System for the 2017 Multi-Genre Broadcast Challenge
In order to achieve a robust ADI system, we explored both Siamese neural network models to learn similarity and dissimilarities among Arabic dialects, as well as i-vector post-processing to adapt domain mismatches.
Unsupervised Domain Adaptation for Robust Speech Recognition via Variational Autoencoder-Based Data Augmentation
Research on robust speech recognition can be regarded as trying to overcome this domain mismatch issue.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Learning Latent Representations for Speech Generation and Transformation
In this paper, we apply a convolutional VAE to model the generative process of natural speech.
What do Neural Machine Translation Models Learn about Morphology?
Neural machine translation (MT) models obtain state-of-the-art performance while maintaining a simple, end-to-end architecture.
Adaptive Bidirectional Backpropagation: Towards Biologically Plausible Error Signal Transmission in Neural Networks
However, it has been argued that this is not biologically plausible because back-propagating error signals with the exact incoming weights are not considered possible in biological neural systems.
Neural Attention for Learning to Rank Questions in Community Question Answering
In real-world data, e. g., from Web forums, text is often contaminated with redundant or irrelevant content, which leads to introducing noise in machine learning algorithms.
Unsupervised Learning of Spoken Language with Visual Context
Humans learn to speak before they can read or write, so why can’t computers do the same?
Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results
Machine translation between Arabic and Hebrew has so far been limited by a lack of parallel corpora, despite the political and cultural importance of this language pair.
A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects
Discriminating between closely-related language varieties is considered a challenging and important task.
The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition
For language modelling, we made available over 110M words crawled from Aljazeera Arabic website Aljazeera. net for a 10 year duration 2000-2011.
Recurrent Neural Network Encoder with Attention for Community Question Answering
We apply a general recurrent neural network (RNN) encoder framework to community question answering (cQA) tasks.
Deep Multimodal Semantic Embeddings for Speech and Images
In this paper, we present a model which takes as input a corpus of images with relevant spoken captions and finds a correspondence between the two modalities.
Highway Long Short-Term Memory RNNs for Distant Speech Recognition
In this paper, we extend the deep long short-term memory (DLSTM) recurrent neural networks by introducing gated direct connections between memory cells in adjacent layers.
Prediction-Adaptation-Correction Recurrent Neural Networks for Low-Resource Language Speech Recognition
In this paper, we investigate the use of prediction-adaptation-correction recurrent neural networks (PAC-RNNs) for low-resource speech recognition.
Automatic Dialect Detection in Arabic Broadcast Speech
We used these features in a binary classifier to discriminate between Modern Standard Arabic (MSA) and Dialectal Arabic, with an accuracy of 100%.
Unsupervised Lexicon Discovery from Acoustic Input
We present a model of unsupervised phonological lexicon discovery{---}the problem of simultaneously learning phoneme-like and word-like units from acoustic input.
