A General and Efficient Training for Transformer via Token Expansion

1 code implementation • 31 Mar 2024 • Wenxuan Huang, Yunhang Shen, Jiao Xie, Baochang Zhang, Gaoqi He, Ke Li, Xing Sun, Shaohui Lin

The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost.

Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler

1 code implementation • 1 Jul 2023 • Shaohui Lin, Wenxuan Huang, Jiao Xie, Baochang Zhang, Yunhang Shen, Zhou Yu, Jungong Han, David Doermann

In this paper, we propose a novel Knowledge-driven Differential Filter Sampler~(KDFS) with Masked Filter Modeling~(MFM) framework for filter pruning, which globally prunes the redundant filters based on the prior knowledge of a pre-trained model in a differential and non-alternative optimization.

Image Classification Network Pruning

Rapid COVID-19 Risk Screening by Eye-region Manifestations

no code implementations • 12 Jun 2021 • Yanwei Fu, Lei Zhao, Haojie Zheng, Qiang Sun, Li Yang, Hong Li, Jiao Xie, xiangyang xue, Feng Li, Yuan Li, Wei Wang, Yantao Pei, Jianmin Wang, Xiuqi Wu, Yanhua Zheng, Hongxia Tian Mengwei Gu1

It is still nontrivial to develop a new fast COVID-19 screening method with the easier access and lower cost, due to the technical and cost limitations of the current testing methods in the medical resource-poor districts.

Ethics

Training convolutional neural networks with cheap convolutions and online distillation

1 code implementation • 28 Sep 2019 • Jiao Xie, Shaohui Lin, Yichen Zhang, Linkai Luo

The large memory and computation consumption in convolutional neural networks (CNNs) has been one of the main barriers for deploying them on resource-limited systems.

Knowledge Distillation