MTL-SLT: Multi-Task Learning for Spoken Language Tasks

no code implementations • NLP4ConvAI (ACL) 2022 • Zhiqi Huang, Milind Rao, Anirudh Raju, Zhe Zhang, Bach Bui, Chul Lee

The proposed framework benefits from three key aspects: 1) pre-trained sub-networks of ASR model and language model; 2) multi-task learning objective to exploit shared knowledge from different tasks; 3) end-to-end training of ASR and downstream NLP task based on sequence loss.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +5

Improving fairness for spoken language understanding in atypical speech with Text-to-Speech

1 code implementation • 16 Nov 2023 • Helin Wang, Venkatesh Ravichandran, Milind Rao, Becky Lammers, Myra Sydnor, Nicholas Maragakis, Ankur A. Butala, Jayne Zhang, Lora Clawson, Victoria Chovaz, Laureano Moro-Velazquez

Spoken language understanding (SLU) systems often exhibit suboptimal performance in processing atypical speech, typically caused by neurological conditions and motor impairments.

Data Augmentation Fairness +2

Federated Representation Learning for Automatic Speech Recognition

no code implementations • 3 Aug 2023 • Guruprasad V Ramesh, Gopinath Chennupati, Milind Rao, Anit Kumar Sahu, Ariya Rastrow, Jasha Droppo

Federated Learning (FL) is a privacy-preserving paradigm, allowing edge devices to learn collaboratively without sharing data.

Automatic Speech Recognition Federated Learning +5

Federated Self-Learning with Weak Supervision for Speech Recognition

no code implementations • 21 Jun 2023 • Milind Rao, Gopinath Chennupati, Gautam Tiwari, Anit Kumar Sahu, Anirudh Raju, Ariya Rastrow, Jasha Droppo

Automatic speech recognition (ASR) models with low-footprint are increasingly being deployed on edge devices for conversational agents, which enhances privacy.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Learning When to Trust Which Teacher for Weakly Supervised ASR

no code implementations • 21 Jun 2023 • Aakriti Agrawal, Milind Rao, Anit Kumar Sahu, Gopinath Chennupati, Andreas Stolcke

We show the efficacy of our approach using LibriSpeech and LibriLight benchmarks and find an improvement of 4 to 25\% over baselines that uniformly weight all the experts, use a single expert model, or combine experts using ROVER.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scale

no code implementations • 19 Jul 2022 • Gopinath Chennupati, Milind Rao, Gurpreet Chadha, Aaron Eakin, Anirudh Raju, Gautam Tiwari, Anit Kumar Sahu, Ariya Rastrow, Jasha Droppo, Andy Oberlin, Buddha Nandanoor, Prahalad Venkataramanan, Zheng Wu, Pankaj Sitpure

For end-to-end automatic speech recognition (ASR) tasks, the absence of human annotated labels along with the need for privacy preserving policies for model building makes it a daunting challenge.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

On joint training with interfaces for spoken language understanding

no code implementations • 30 Jun 2021 • Anirudh Raju, Milind Rao, Gautam Tiwari, Pranav Dheram, Bryan Anderson, Zhe Zhang, Chul Lee, Bach Bui, Ariya Rastrow

Spoken language understanding (SLU) systems extract both text transcripts and semantics associated with intents and slots from input speech utterances.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-End

no code implementations • 14 May 2021 • Swayambhu Nath Ray, Minhua Wu, Anirudh Raju, Pegah Ghahremani, Raghavendra Bilgi, Milind Rao, Harish Arsikere, Ariya Rastrow, Andreas Stolcke, Jasha Droppo

On the other hand, a streaming system using per-frame intent posteriors as extra inputs for the RNN-T ASR system yields a 3. 33% relative WERR.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understanding

no code implementations • 12 Feb 2021 • Milind Rao, Pranav Dheram, Gautam Tiwari, Anirudh Raju, Jasha Droppo, Ariya Rastrow, Andreas Stolcke

Spoken language understanding (SLU) systems extract transcriptions, as well as semantics of intent or named entities from speech, and are essential components of voice activated systems.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfaces

no code implementations • 14 Aug 2020 • Milind Rao, Anirudh Raju, Pranav Dheram, Bach Bui, Ariya Rastrow

Finally, we contrast these methods to a jointly trained end-to-end joint SLU model, consisting of ASR and NLU subsystems which are connected by a neural network based interface instead of text, that produces transcripts as well as NLU interpretation.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Distributed Convex Optimization With Limited Communications

no code implementations • 29 Oct 2018 • Milind Rao, Stefano Rini, Andrea Goldsmith

In this paper, a distributed convex optimization algorithm, termed \emph{distributed coordinate dual averaging} (DCDA) algorithm, is proposed.

Distributed Optimization valid

Deep Learning for Joint Source-Channel Coding of Text

1 code implementation • 19 Feb 2018 • Nariman Farsad, Milind Rao, Andrea Goldsmith

We consider the problem of joint source and channel coding of structured data such as natural language over a noisy channel.