Lightweight Self-Knowledge Distillation with Multi-source Information Fusion

1 code implementation • 16 May 2023 • Xucong Wang, Pengchao Han, Lei Guo

Specifically, we introduce a Distillation with Reverse Guidance (DRG) method that considers different levels of information extracted by the model, including edge, shape, and detail of the input data, to construct a more informative teacher.

Self-Knowledge Distillation