Search Results for author:
Found 27 papers, 5 papers with code
CALM: Contrastive Aligned Audio-Language Multirate and Multimodal Representations
By aligning audio representations to pretrained language representations and utilizing contrastive information between acoustic inputs, CALM is able to bootstrap audio embedding competitive with existing audio representation models in only a few hours of training time.
Analysis and Tuning of a Voice Assistant System for Dysfluent Speech
Dysfluencies and variations in speech pronunciation can severely degrade speech recognition performance, and for many individuals with moderate-to-severe speech disorders, voice operated systems do not work.
Unsupervised Speech Representation Learning for Behavior Modeling using Triplet Enhanced Contextualized Networks
In this paper, we exploit the stationary properties of human behavior within an interaction and present a representation learning method to capture behavioral information from speech in an unsupervised way.
Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies
With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care to assist in training, supervision, and quality assurance of services.
Confusion2vec 2.0: Enriching Ambiguous Spoken Language Representations with Subwords
Confusion2vec, motivated from human speech production and perception, is a word vector representation which encodes ambiguities present in human spoken language in addition to semantics and syntactic information.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Speaker Diarization with Lexical Information
This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
An analysis of observation length requirements for machine understanding of human behaviors from spoken language
In this paper, we investigate this link and present an analysis framework that determines appropriate window lengths for the task of behavior estimation.
Speaker-invariant Affective Representation Learning via Adversarial Training
In this paper, we propose a machine learning framework to obtain speech emotion representations by limiting the effect of speaker variability in the speech signals.
RNN based Incremental Online Spoken Language Understanding
We introduce and analyze different recurrent neural network architectures for incremental and online processing of the ASR transcripts and compare it to the existing offline systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Linking emotions to behaviors through deep transfer learning
Further, we investigate the importance of emotional-context in the expression of behavior by constraining (or not) the neural networks' contextual view of the data.
Multimodal Embeddings from Language Models
Word embeddings such as ELMo have recently been shown to model word semantics with greater efficacy through contextualized learning on large-scale language corpora, resulting in significant improvement in state of the art across many natural language tasks.
Behavior Gated Language Models
In this work we derive motivation from psycholinguistics and propose the addition of behavioral information into the context of language modeling.
Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language
Cancer impacts the quality of life of those diagnosed as well as their spouse caregivers, in addition to potentially influencing their day-to-day behaviors.
Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distance
We find that our proposed measure is correlated with the therapist's empathy towards their patient in Motivational Interviewing and with affective behaviors in Couples Therapy.
Spoken Language Intent Detection using Confusion2Vec
In this paper, we address the spoken language intent detection under noisy conditions imposed by automatic speech recognition (ASR) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speaker Diarization With Lexical Information
This work presents a novel approach to leverage lexical information for speaker diarization.
Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities
In this paper, we propose a novel word vector representation, Confusion2Vec, motivated from the human speech production and perception that encodes representational ambiguity.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multi-label Multi-task Deep Learning for Behavioral Coding
We propose a methodology for estimating human behaviors in psychotherapy sessions using mutli-label and multi-task learning paradigms.
Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings
Unsupervised learning has been an attractive method for easily deriving meaningful data representations from vast amounts of unlabeled data.
Modeling Interpersonal Influence of Verbal Behavior in Couples Therapy Dyadic Interactions
Dyadic interactions among humans are marked by speakers continuously influencing and reacting to each other in terms of responses and behaviors, among others.
Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations
Evaluations are presented on (i) comparisons of earlier GMM-HMM and the newer DNN Models, (ii) effectiveness of standard adaptation techniques versus transfer learning, (iii) various adaptation configurations in tackling the variabilities present in children speech, in terms of (a) acoustic spectral variability, and (b) pronunciation variability and linguistic constraints.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards an Unsupervised Entrainment Distance in Conversational Speech using Deep Neural Networks
Entrainment is a known adaptation mechanism that causes interaction participants to adapt or synchronize their acoustic characteristics.
Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics
Two sets of experiments are done in different scenarios to evaluate the strength of NPC embeddings and compare with state-of-the-art in-domain supervised methods.
Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context Modeling
In this work we model ASR as a phrase-based noisy transformation channel and propose an error correction system that can learn from the aggregate errors of all the independent modules constituting the ASR and attempt to invert those.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Unsupervised Latent Behavior Manifold Learning from Acoustic Features: audio2behavior
Behavioral annotation using signal processing and machine learning is highly dependent on training data and manual annotations of behavioral labels.
Sparsely Connected and Disjointly Trained Deep Neural Networks for Low Resource Behavioral Annotation: Acoustic Classification in Couples' Therapy
We propose a Sparsely-Connected and Disjointly-Trained DNN (SD-DNN) framework to deal with limited data.
