Search Results for author:
Found 59 papers, 9 papers with code
Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR
Contextual biasing enables speech recognizers to transcribe important phrases in the speaker's context, such as contact names, even if they are rare in, or absent from, the training data.
TransformerFAM: Feedback attention is working memory
While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs.
Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models
In the present work, we study one such strategy: applying multiple frame reduction layers in the encoder to compress encoder outputs into a small number of output frames.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models
We conducted extensive experiments with a 2-billion parameter USM on a large-scale voice search dataset to evaluate our proposed method.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Contextual Biasing with the Knuth-Morris-Pratt Matching Algorithm
Contextual biasing refers to the problem of biasing the automatic speech recognition (ASR) systems towards rare entities that are relevant to the specific user or application scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Massive End-to-end Models for Short Search Queries
In this work, we investigate two popular end-to-end automatic speech recognition (ASR) models, namely Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), for offline recognition of voice search queries, with up to 2B model parameters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network
Whether it is the conventional modularized approach or the more recent end-to-end neural diarization (EEND), an additional automatic speech recognition (ASR) model and an orchestration algorithm are required to associate the speaker labels with recognized words.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Augmenting conformers with structured state-space sequence models for online speech recognition
Online speech recognition, where the model only accesses context to the left, is an important and challenging use case for ASR systems.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
Text Injection for Capitalization and Turn-Taking Prediction in Speech Models
Text injection for automatic speech recognition (ASR), wherein unpaired text-only data is used to supplement paired audio-text data, has shown promising improvements for word error rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR
Conformer models maintain a large number of internal states, the vast majority of which are associated with self-attention layers.
JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition
We propose JEIT, a joint end-to-end (E2E) model and internal language model (ILM) training method to inject large-scale unpaired text into ILM during E2E training which improves rare-word speech recognition.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Improving Deliberation by Text-Only and Semi-Supervised Training
Text-only and semi-supervised training based on audio-only data has gained popularity recently due to the wide availability of unlabeled text and speech data.
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Streaming Align-Refine for Non-autoregressive Deliberation
We propose a streaming non-autoregressive (non-AR) decoding algorithm to deliberate the hypothesis alignment of a streaming RNN-T model.
Improving Rare Word Recognition with LM-aware MWER Training
Language models (LMs) significantly improve the recognition accuracy of end-to-end (E2E) models on words rarely seen during training, when used in either the shallow fusion or the rescoring setups.
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Contrastively Disentangled Sequential Variational Autoencoder
We propose a novel sequence representation learning method, named Contrastively Disentangled Sequential Variational Autoencoder (C-DSVAE), to extract and separate the static (time-invariant) and dynamic (time-variant) factors in the latent space.
Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspective
Under this model, latent correlation maximization is shown to guarantee the extraction of the shared components across views (up to certain ambiguities).
Representation Learning for Sequence Data with Deep Autoencoding Predictive Components
We propose Deep Autoencoding Predictive Components (DAPC) -- a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
An investigation of phone-based subword units for end-to-end speech recognition
For Switchboard, our phone-based BPE system achieves 6. 8\%/14. 4\% word error rate (WER) on the Switchboard/CallHome portion of the test set while joint decoding achieves 6. 3\%/13. 3\% WER.
Data Techniques For Online End-to-end Speech Recognition
Practitioners often need to build ASR systems for new use cases in a short amount of time, given limited in-domain data.
Semi-supervised ASR by End-to-end Self-training
While deep learning based end-to-end automatic speech recognition (ASR) systems have greatly simplified modeling pipelines, they suffer from the data sparsity issue.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Acoustic scene analysis with multi-head attention networks
Acoustic Scene Classification (ASC) is a challenging task, as a single scene may involve multiple events that contain complex sound patterns.
Multimodal and Multi-view Models for Emotion Recognition
Studies on emotion recognition (ER) show that combining lexical and acoustic information results in more robust and accurate models.
Variational recurrent models for representation learning
Existing variational recurrent models typically use stochastic recurrent connections to model the dependence among neighboring latent variables, while generation assumes independence of generated data per time step given the latent sequence.
Everything old is new again: A multi-view learning approach to learning using privileged information and distillation
We adopt a multi-view approach for analyzing two knowledge transfer settings---learning using privileged information (LUPI) and distillation---in a common framework.
Acoustic feature learning using cross-domain articulatory measurements
Previous work has shown that it is possible to improve speech recognition by learning acoustic features from paired acoustic-articulatory data, for example by using canonical correlation analysis (CCA) or its deep extensions.
Distributed Stochastic Multi-Task Learning with Graph Regularization
We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines.
Stochastic Nonconvex Optimization with Large Minibatches
We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks.
Acoustic Feature Learning via Deep Variational Canonical Correlation Analysis
We study the problem of acoustic feature learning in the setting where we have access to another (non-acoustic) modality for feature learning but not at test time.
Efficient coordinate-wise leading eigenvector computation
We develop and analyze efficient "coordinate-wise" methods for finding the leading eigenvector, where each step involves only a vector-vector product.
Memory and Communication Efficient Distributed Stochastic Optimization with Minibatch-Prox
We present and analyze an approach for distributed stochastic optimization which is statistically optimal and achieves near-linear speedups (up to logarithmic factors).
Stochastic Canonical Correlation Analysis
We study the sample complexity of canonical correlation analysis (CCA), \ie, the number of samples needed to estimate the population canonical correlation and directions up to arbitrarily small error.
Multi-view Recurrent Neural Acoustic Word Embeddings
Recent work has begun exploring neural acoustic word embeddings---fixed-dimensional vector representations of arbitrary-length speech segments corresponding to words.
End-to-End Training Approaches for Discriminative Segmental Models
Similarly to hybrid HMM-neural network models, segmental models of this class can be trained in two stages (frame classifier training followed by linear segmental model weight training), end to end (joint training of both frame classifier and linear weights), or with end-to-end fine-tuning after two-stage training.
Deep Variational Canonical Correlation Analysis
We present deep variational canonical correlation analysis (VCCA), a deep multi-view learning model that extends the latent variable model interpretation of linear CCA to nonlinear observation models parameterized by deep neural networks.
Lexicon-Free Fingerspelling Recognition from Video: Data, Models, and Signer Adaptation
Recognizing fingerspelling is challenging for a number of reasons: It involves quick, small motions that are often highly coarticulated; it exhibits significant variation between signers; and there has been a dearth of continuous fingerspelling data collected.
Efficient Segmental Cascades for Speech Recognition
Discriminative segmental models offer a way to incorporate flexible feature functions into speech recognition.
Efficient Globally Convergent Stochastic Optimization for Canonical Correlation Analysis
We study the stochastic optimization of canonical correlation analysis (CCA), whose objective is nonconvex and does not decouple over training samples.
Signer-independent Fingerspelling Recognition with Deep Neural Network Adaptation
Previous work has shown that it is possible to achieve almost 90% accuracies on fingerspelling recognition in a signer-dependent setting.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Network Inference by Learned Node-Specific Degree Prior
We propose a novel method for network inference from partially observed edges using a node-specific degree prior.
On Column Selection in Approximate Kernel Canonical Correlation Analysis
We study the problem of column selection in large-scale kernel canonical correlation analysis (KCCA) using the Nystr\"om approximation, where one approximates two positive semi-definite kernel matrices using "landmark" points from the training set.
On Deep Multi-View Representation Learning: Objectives and Optimization
We consider learning representations (features) in the setting in which we have access to multiple unlabeled views of the data for learning while only one view is available for downstream tasks.
Nonparametric Canonical Correlation Analysis
Several nonlinear extensions of the original linear CCA have been proposed, including kernel and deep neural network methods.
Large-Scale Approximate Kernel Canonical Correlation Analysis
Kernel canonical correlation analysis (KCCA) is a nonlinear multi-view representation learning technique with broad applicability in statistics and machine learning.
Stochastic Optimization for Deep CCA via Nonlinear Orthogonal Iterations
Deep CCA is a recently proposed deep neural network extension to the traditional canonical correlation analysis (CCA), and has been successful for multi-view representation learning in several domains.
Deep convolutional acoustic word embeddings using word-pair side information
Recent studies have been revisiting whole words as the basic modelling unit in speech recognition and query applications, instead of phonetic units.
Discriminative Segmental Cascades for Feature-Rich Phone Recognition
A typical solution is to use approximate decoding, either by beam pruning in a single pass or by beam pruning to generate a lattice followed by a second pass.
Projection onto the capped simplex
We provide a simple and efficient algorithm for computing the Euclidean projection of a point onto the capped simplex---a simplex with an additional uniform bound on each coordinate---together with an elementary proof.
An $\mathcal{O}(n\log n)$ projection operator for weighted $\ell_1$-norm regularization with sum constraint
We provide a simple and efficient algorithm for the projection operator for weighted $\ell_1$-norm regularization subject to a sum constraint, together with an elementary proof.
The Laplacian K-modes algorithm for clustering
In addition to finding meaningful clusters, centroid-based clustering algorithms such as K-means or mean-shift should ideally find centroids that are valid patterns in the input space, representative of data in their cluster.
The role of dimensionality reduction in linear classification
Using the method of auxiliary coordinates, we give a simple, efficient algorithm to train a combination of nonlinear DR and a classifier, and apply it to a RBF mapping with a linear SVM.
LASS: a simple assignment model with Laplacian smoothing
We consider the problem of learning soft assignments of $N$ items to $K$ categories given two sources of information: an item-category similarity matrix, which encourages items to be assigned to categories they are similar to (and to not be assigned to categories they are dissimilar to), and an item-item similarity matrix, which encourages similar items to have similar assignments.
Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application
We provide an elementary proof of a simple, efficient algorithm for computing the Euclidean projection of a point onto the probability simplex.
The K-modes algorithm for clustering
Many clustering algorithms exist that estimate a cluster centroid, such as K-means, K-medoids or mean-shift, but no algorithm seems to exist that clusters data by returning exactly K meaningful modes.
A Denoising View of Matrix Completion
In matrix completion, we are given a matrix where the values of only some of the entries are present, and we want to reconstruct the missing ones.
