Search Results for author:
Found 49 papers, 4 papers with code
Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion
We propose an approach for continuous prediction of turn-taking and backchanneling locations in spoken dialogue by fusing a neural acoustic model with a large language model (LLM).
Post-Training Embedding Alignment for Decoupling Enrollment and Runtime Speaker Recognition Models
Automated speaker identification (SID) is a crucial step for the personalization of a wide range of speech-enabled services.
Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition
The use of low-rank adaptation (LoRA) with frozen pretrained language models (PLMs) has become increasing popular as a mainstream, resource-efficient modeling approach for memory-constrained hardware.
Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with transcribed speech instead of conventional written text.
Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue
Specifically, our framework serializes tasks in the order of current paralinguistic attribute prediction, response paralinguistic attribute prediction, and response text generation with autoregressive conditioning.
Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting
We explore the ability of large language models (LLMs) to act as speech recognition post-processors that perform rescoring and error correction.
Ranked #3 on
Speech Recognition
on WSJ eval92
(using extra training data)
Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition
We propose a neural language modeling system based on low-rank adaptation (LoRA) for speech recognition output rescoring.
Learning When to Trust Which Teacher for Weakly Supervised ASR
We show the efficacy of our approach using LibriSpeech and LibriLight benchmarks and find an improvement of 4 to 25\% over baselines that uniformly weight all the experts, use a single expert model, or combine experts using ROVER.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Streaming Speech-to-Confusion Network Speech Recognition
In interactive automatic speech recognition (ASR) systems, low-latency requirements limit the amount of search space that can be explored during decoding, particularly in end-to-end neural ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers
End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names and places.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Cross-utterance ASR Rescoring with Graph-based Label Propagation
We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity.
Adaptive Endpointing with Deep Contextual Multi-armed Bandits
In this paper, we aim to provide a solution for adaptive endpointing by proposing an efficient method for choosing an optimal endpointing configuration given utterance-level audio features in an online setting, while avoiding hyperparameter grid-search.
Stutter-TTS: Controlled Synthesis and Improved Recognition of Stuttered Speech
By fine-tuning an ASR model on synthetic stuttered speech we are able to reduce word error by 5. 7% relative on stuttered utterances, with only minor (<0. 2% relative) degradation for fluent utterances.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition
We evaluate three end-to-end deep models, including LAS, hybrid CTC/attention, and RNN transducer, on the open-source LibriSpeech and TIMIT corpora.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities
As for other forms of AI, speech recognition has recently been examined with respect to performance disparities across different user cohorts.
Reducing Geographic Disparities in Automatic Speech Recognition via Elastic Weight Consolidation
A popular approach is to fine-tune the model with data from regions where the ASR model has a higher word error rate (WER).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Adversarial Reweighting for Speaker Verification Fairness
Results show that the pairwise weighting method can achieve 1. 08% overall EER, 1. 25% for male and 0. 67% for female speakers, with relative EER reductions of 7. 7%, 10. 1% and 3. 0%, respectively.
Graph-based Multi-View Fusion and Local Adaptation: Mitigating Within-Household Confusability for Speaker Identification
Speaker identification (SID) in the household scenario (e. g., for smart speakers) is an important but challenging problem due to limited number of labeled (enrollment) utterances, confusable voices, and demographic imbalances.
CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals
Training the model initially with proxy context retains 67% of the perplexity gain after adapting to real context.
openFEAT: Improving Speaker Identification by Open-set Few-shot Embedding Adaptation with Transformer
Household speaker identification with few enrollment utterances is an important yet challenging problem, especially when household members share similar voice characteristics and room acoustics.
Improving fairness in speaker verification via Group-adapted Fusion Network
This is observed especially with underrepresented demographic groups sharing similar voice characteristics.
Contrastive-mixup learning for improved speaker verification
In this work, we propose contrastive-mixup, a novel augmentation strategy that learns distinguishing representations based on a distance metric.
Mitigating Closed-model Adversarial Examples with Bayesian Neural Modeling for Enhanced End-to-End Speech Recognition
In this work, we aim to enhance the system robustness of end-to-end automatic speech recognition (ASR) against adversarially-noisy speech examples.
Self-supervised Speaker Recognition Training Using Human-Machine Dialogues
Speaker recognition, recognizing speaker identities based on voice alone, enables important downstream applications, such as personalization and authentication.
ASR-Aware End-to-end Neural Diarization
We present a Conformer-based end-to-end neural diarization (EEND) model that uses both acoustic input and features derived from an automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Specifically, training a bidirectional model like BERT on a discriminative objective such as minimum WER (MWER) has not been explored.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Improving Speaker Identification for Shared Devices by Adapting Embeddings to Speaker Subsets
Second, a scoring function is applied between a runtime utterance and each speaker profile.
Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition
Based on whether the speech content is constrained or not, both text-dependent (TD) and text-independent (TI) speaker recognition models may be used.
Graph-based Label Propagation for Semi-Supervised Speaker Identification
We show in experiments on the VoxCeleb dataset that this approach makes effective use of unlabeled data and improves speaker identification accuracy compared to two state-of-the-art scoring methods as well as their semi-supervised variants based on pseudo-labels.
End-to-end Neural Diarization: From Transformer to Conformer
We propose a new end-to-end neural diarization (EEND) system that is based on Conformer, a recently proposed neural architecture that combines convolutional mappings and Transformer to model both local and global dependencies in speech.
Attention-based Contextual Language Model Adaptation for Speech Recognition
When applied to a large de-identified dataset of utterances collected by a popular voice assistant platform, our method reduces perplexity by 7. 0% relative over a standard LM that does not incorporate contextual information.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-End
On the other hand, a streaming system using per-frame intent posteriors as extra inputs for the RNN-T ASR system yields a 3. 33% relative WERR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Reranking Machine Translation Hypotheses with Structured and Web-based Language Models
In this paper, we investigate the use of linguistically motivated and computationally efficient structured language models for reranking N-best hypotheses in a statistical machine translation system.
Wav2vec-C: A Self-supervised Model for Speech Representation Learning
However, the quantization process is regularized by an additional consistency network that learns to reconstruct the input features to the wav2vec 2. 0 network from the quantized representations in a way similar to a VQ-VAE model.
Personalization Strategies for End-to-End Speech Recognition Systems
We also describe a novel second-pass de-biasing approach: used in conjunction with a first-pass shallow fusion that optimizes on oracle WER, we can achieve an additional 14% improvement on personalized content recognition, and even improve accuracy for the general use case by up to 2. 5%.
Contrastive Unsupervised Learning for Speech Emotion Recognition
Speech emotion recognition (SER) is a key technology to enable more natural human-machine communication.
Ranked #2 on
Speech Emotion Recognition
on MSP-Podcast (Dominance)
(using extra training data)
Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understanding
Spoken language understanding (SLU) systems extract transcriptions, as well as semantics of intent or named entities from speech, and are essential components of voice activated systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling
Accents mismatching is a critical problem for end-to-end ASR.
BW-EDA-EEND: Streaming End-to-End Neural Speaker Diarization for a Variable Number of Speakers
We present a novel online end-to-end neural diarization system, BW-EDA-EEND, that processes data incrementally for a variable number of speakers.
DOVER-Lap: A Method for Combining Overlap-aware Diarization Outputs
Several advances have been made recently towards handling overlapping speech for speaker diarization.
Audio and Speech Processing Sound
Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognition
Unlike previous work on this topic, which performs on-the-fly limited-size beam-search decoding and generates alignment scores for expected edit-distance computation, in our proposed method, we re-calculate and sum scores of all the possible alignments for each hypothesis in N-best lists.
Combining Acoustics, Content and Interaction Features to Find Hot Spots in Meetings
Involvement hot spots have been proposed as a useful concept for meeting analysis and studied off and on for over 15 years.
Improving Diarization Robustness using Diversification, Randomization and the DOVER Algorithm
Speaker diarization based on bottom-up clustering of speech segments by acoustic similarity is often highly sensitive to the choice of hyperparameters, such as the initial number of clusters and feature weighting.
DOVER: A Method for Combining Diarization Outputs
Speech recognition and other natural language tasks have long benefited from voting-based algorithms as a method to aggregate outputs from several systems to achieve a higher accuracy than any of the individual systems.
Session-level Language Modeling for Conversational Speech
We propose to generalize language models for conversational speech recognition to allow them to operate across utterance boundaries and speaker changes, thereby capturing conversation-level phenomena such as adjacency pairs, lexical entrainment, and topical coherence.
Comparing Human and Machine Errors in Conversational Speech Transcription
In this paper we approach this question by comparing the output of our most accurate CTS recognition system to that of a standard speech transcription vendor pipeline.
