Application of Knowledge Distillation to Multi-task Speech Representation Learning

no code implementations • 29 Oct 2022 • Mine Kerpicci, Van Nguyen, Shuhua Zhang, Erik Visser

Model architectures such as wav2vec 2. 0 and HuBERT have been proposed to learn speech representations from audio waveforms in a self-supervised manner.

Keyword Spotting Knowledge Distillation +4