Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation

1 code implementation • 13 Jun 2022 • Zengyu Qiu, Xinzhu Ma, Kunlin Yang, Chunya Liu, Jun Hou, Shuai Yi, Wanli Ouyang

Besides, our DPK makes the performance of the student model positively correlated with that of the teacher model, which means that we can further boost the accuracy of students by applying larger teachers.

Image Classification Knowledge Distillation +3