Speculative Streaming: Fast LLM Inference without Auxiliary Models

no code implementations • 16 Feb 2024 • Nikhil Bhendawade, Irina Belousova, Qichen Fu, Henry Mason, Mohammad Rastegari, Mahyar Najibi

Speculative decoding is a prominent technique to speed up the inference of a large target language model based on predictions of an auxiliary draft model.

Language Modeling Language Modelling

Conformer-Based Speech Recognition On Extreme Edge-Computing Devices

no code implementations • 16 Dec 2023 • MingBin Xu, Alex Jin, Sicheng Wang, Mu Su, Tim Ng, Henry Mason, Shiyi Han, Zhihong Lei, Yaqiao Deng, Zhen Huang, Mahesh Krishnamoorthy

With increasingly more powerful compute capabilities and resources in today's devices, traditionally compute-intensive automatic speech recognition (ASR) has been moving from the cloud to devices to better protect user privacy.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Variable Attention Masking for Configurable Transformer Transducer Speech Recognition

no code implementations • 2 Nov 2022 • Pawel Swietojanski, Stefan Braun, Dogan Can, Thiago Fraga da Silva, Arnab Ghoshal, Takaaki Hori, Roger Hsiao, Henry Mason, Erik McDermott, Honza Silovsky, Ruchir Travadi, Xiaodan Zhuang

This work studies the use of attention masking in transformer transducer based speech recognition for building a single configurable model for different deployment scenarios.

speech-recognition Speech Recognition

Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation

no code implementations • 21 Oct 2022 • Thien Nguyen, Nathalie Tran, Liuhui Deng, Thiago Fraga da Silva, Matthew Radzihovsky, Roger Hsiao, Henry Mason, Stefan Braun, Erik McDermott, Dogan Can, Pawel Swietojanski, Lyan Verwimp, Sibel Oyman, Tresi Arvizo, Honza Silovsky, Arnab Ghoshal, Mathieu Martel, Bharat Ram Ambati, Mohamed Ali

Code-switching describes the practice of using more than one language in the same sentence.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications

no code implementations • 16 Feb 2021 • Matthias Paulik, Matt Seigel, Henry Mason, Dominic Telaar, Joris Kluivers, Rogier Van Dalen, Chi Wai Lau, Luke Carlson, Filip Granqvist, Chris Vandevelde, Sudeep Agarwal, Julien Freudiger, Andrew Byde, Abhishek Bhowmick, Gaurav Kapoor, Si Beaumont, Áine Cahill, Dominic Hughes, Omid Javidbakht, Fei Dong, Rehan Rishi, Stanley Hung

We describe the design of our federated task processing system.

Federated Learning

Voice trigger detection from LVCSR hypothesis lattices using bidirectional lattice recurrent neural networks

no code implementations • 29 Feb 2020 • Woojay Jeon, Leo Liu, Henry Mason

We propose a method to reduce false voice triggers of a speech-enabled personal assistant by post-processing the hypothesis lattice of a server-side large-vocabulary continuous speech recognizer (LVCSR) via a neural network.

SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognition

no code implementations • 4 Oct 2019 • Zhen Huang, Tim Ng, Leo Liu, Henry Mason, Xiaodan Zhuang, Daben Liu

The most popular way to train very deep CNNs is to use shortcut connections (SC) together with batch normalization (BN).

Inference Optimization speech-recognition +1