Search Results for author:
Found 19 papers, 1 papers with code
Learning Visual Composition through Improved Semantic Guidance
One can observe this limitation in representations learned through captions or contrastive learning -- where the learned model treats an image essentially as a bag of words.
Knowledge Graph Reasoning with Self-supervised Reinforcement Learning
To alleviate the distributional mismatch issue in general self-supervised RL (SSRL), in our supervised learning (SL) stage, the agent selects actions based on the policy network and learns from generated labels; this self-generation of labels is the intuition behind the name self-supervised.
Retrieval Augmented End-to-End Spoken Dialog Models
We recently developed SLM, a joint speech and language model, which fuses a pretrained foundational speech model and a large language model (LLM), while preserving the in-context learning capability intrinsic to the pretrained LLM.
Detecting Speech Abnormalities with a Perceiver-based Sequence Classifier that Leverages a Universal Speech Model
We propose a Perceiver-based sequence classifier to detect abnormalities in speech reflective of several neurological disorders.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SLM: Bridge the thin gap between speech and text foundation models
We present a joint Speech and Language Model (SLM), a multitask, multilingual, and dual-modal model that takes advantage of pretrained foundational speech and language models.
Efficient Adapters for Giant Speech Models
However, finetuning all parameters from the self-supervised learned model can be computationally expensive, and becomes infeasiable as the size of the model and the number of downstream tasks scales.
Speech-to-Text Adapter and Speech-to-Entity Retriever Augmented LLMs for Speech Understanding
Large Language Models (LLMs) have been applied in the speech domain, often incurring a performance drop due to misaligned between speech and language representations.
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
AnyTOD: A Programmable Task-Oriented Dialog System
We propose AnyTOD, an end-to-end, zero-shot task-oriented dialog (TOD) system capable of handling unseen tasks without task-specific training.
Speech Aware Dialog System Technology Challenge (DSTC11)
The research on this topic is stymied by the lack of a public corpus.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Knowledge-grounded Dialog State Tracking
Knowledge (including structured knowledge such as schema and ontology, and unstructured knowledge such as web corpus) is a critical part of dialog understanding, especially for unseen tasks and domains.
Unsupervised Slot Schema Induction for Task-oriented Dialog
Carefully-designed schemas describing how to collect and annotate dialog corpora are a prerequisite towards building task-oriented dialog systems.
RNN Transducers for Nested Named Entity Recognition with constraints on alignment for long sequences
Through empirical experiments on a challenging real-world medical NER task with multiple nested ontologies, we demonstrate that our fixed alignment model outperforms the standard RNN-T model, improving F1-score from 0. 70 to 0. 74.
Word-level confidence estimation for RNN transducers
Confidence estimate is an often requested feature in applications such as medical transcription where errors can impact patient care and the confidence estimate could be used to alert medical professionals to verify potential errors in recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Understanding Medical Conversations: Rich Transcription, Confidence Scores & Information Extraction
Our transformer-based streaming model performs at about 20% WER on the ASR task, 6% WDER on the diarization task, 43% SER on periods, 52% SER on commas, 43% SER on question marks and 30% SER on capitalization.
The Medical Scribe: Corpus Development and Model Performance Analyses
We used this annotation scheme to label a corpus of about 6k clinical encounters.
Joint Speech Recognition and Speaker Diarization via Sequence Transduction
The task of assigning words to speakers is typically addressed by merging the outputs of two separate systems, namely, an automatic speech recognition (ASR) system and a speaker diarization (SD) system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
We present results that show it is possible to build a competitive, greatly simplified, large vocabulary continuous speech recognition system with whole words as acoustic units.
Improvements to deep convolutional neural networks for LVCSR
We find that with these improvements, particularly with fMLLR and dropout, we are able to achieve an additional 2-3% relative improvement in WER on a 50-hour Broadcast News task over our previous best CNN baseline.
