Search Results for author:
Found 8 papers, 0 papers with code
Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation
Intermediate layer matching is shown as an effective approach for improving knowledge distillation (KD).
Conformer with dual-mode chunked attention for joint online and offline ASR
In the dual-mode Conformer Transducer model, layers can function in online or offline mode while sharing parameters, and in-place knowledge distillation from offline to online mode is applied in training to improve online accuracy.
CILDA: Contrastive Data Augmentation using Intermediate Layer Knowledge Distillation
Knowledge distillation (KD) is an efficient framework for compressing large-scale pre-trained language models.
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
To address these problems, we propose a RAndom Intermediate Layer Knowledge Distillation (RAIL-KD) approach in which, intermediate layers from the teacher model are selected randomly to be distilled into the intermediate layers of the student model.
Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning with Self-Knowledge Distillation
End-to-end automatic speech recognition (ASR), unlike conventional ASR, does not have modules to learn the semantic representation from speech encoder.
Ranked #12 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networks
In this paper, we introduce a novel framework for fine-tuning a pre-trained ASR model using the GAN objective where the ASR model acts as a generator and a discriminator tries to distinguish the ASR output from the real data.
Distilled embedding: non-linear embedding factorization using knowledge distillation
Word-embeddings are a vital component of Natural Language Processing (NLP) systems and have been extensively researched.
Semi-Supervised Regression with Generative Adverserial Networks for End to End Learning in Autonomous Driving
We performed several experiments on a publicly available driving dataset to evaluate our proposed method, and the results are very promising.
