Understanding the Success of Knowledge Distillation -- A Data Augmentation Perspective

no code implementations • 29 Sep 2021 • Huan Wang, Suhas Lohit, Michael Jeffrey Jones, Yun Fu

We achieve new state-of-the-art accuracy by using the original KD loss armed with stronger augmentation schemes, compared to existing state-of-the-art methods that employ more advanced distillation losses.

Active Learning Data Augmentation +1