Knowledge Distillation with the Reused Teacher Classifier

1 code implementation • 26 Mar 2022 • Defang Chen, Jian-Ping Mei, Hailin Zhang, Can Wang, Yan Feng, Chun Chen

Knowledge distillation aims to compress a powerful yet cumbersome teacher model into a lightweight student model without much sacrifice of performance.

Knowledge Distillation

Confidence-Aware Multi-Teacher Knowledge Distillation

1 code implementation • 30 Dec 2021 • Hailin Zhang, Defang Chen, Can Wang

Knowledge distillation is initially introduced to utilize additional supervision from a single teacher model for the student model training.

Knowledge Distillation Transfer Learning

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework

1 code implementation • 14 Dec 2021 • Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, Bin Cui

Embedding models have been an effective learning paradigm for high-dimensional data.

Coexistence under hierarchical resource exploitation: the role of R*-preemption tradeoff

no code implementations • 22 Aug 2019 • Man Qi, Niv DeMalach, Tao Sun, Hailin Zhang

Thus, we developed an extension of resource competition theory to investigate partial and total preemption (in the latter, the preemptor is unaffected by species with lower preemption rank).