Learn from Balance: Rectifying Knowledge Transfer for Long-Tailed Scenarios

no code implementations • 12 Sep 2024 • Xinlei Huang, Jialiang Tang, Xubin Zheng, Jinjia Zhou, Wenxin Yu, Ning Jiang

Consequently, the teacher network can provide balanced and accurate knowledge to train a reliable student network.

Knowledge Distillation Transfer Learning

Direct Distillation between Different Domains

no code implementations • 12 Jan 2024 • Jialiang Tang, Shuo Chen, Gang Niu, Hongyuan Zhu, Joey Tianyi Zhou, Chen Gong, Masashi Sugiyama

Then, we build a fusion-activation mechanism to transfer the valuable domain-invariant knowledge to the student network, while simultaneously encouraging the adapter within the teacher network to learn the domain-specific knowledge of the target data.

Domain Adaptation Knowledge Distillation

Distribution Shift Matters for Knowledge Distillation with Webly Collected Images

no code implementations • ICCV 2023 • Jialiang Tang, Shuo Chen, Gang Niu, Masashi Sugiyama, Chen Gong

Knowledge distillation aims to learn a lightweight student network from a pre-trained teacher network.

Contrastive Learning Data-free Knowledge Distillation +2