no code implementations • 22 Jul 2023 • Yuwen Zhai, Jing Hao, Liang Gao, Xinyu Li, Yiping Gao, Shumin Han
The hybrid model of self-attention and convolution is one of the methods to lighten ViT.
1 code implementation • 1 Sep 2022 • Chen Sun, Liang Gao, Xinyu Li, Yiping Gao
The proposed DKAN method follows a pretraining-finetuning transfer learning paradigm and a knowledge distillation framework is designed for fine-tuning.