It's All in the Head: Representation Knowledge Distillation through Classifier Sharing

1 code implementation • 18 Jan 2022 • Emanuel Ben-Baruch, Matan Karklinsky, Yossi Biton, Avi Ben-Cohen, Hussam Lawen, Nadav Zamir

Such direct methods may be limited in transferring high-order dependencies embedded in the representation vectors, or in handling the capacity gap between the teacher and student models.

Classification Face Verification +3