Search Results for author:
Found 5 papers, 4 papers with code
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation
Vision Transformers (ViTs) have revolutionized the field of computer vision, yet their deployments on resource-constrained devices remain challenging due to high computational demands.
Ranked #187 on
Image Classification
on ImageNet
Understanding the Effects of Projectors in Knowledge Distillation
Interestingly, we discovered that even if the student and the teacher have the same feature dimensions, adding a projector still helps to improve the distillation performance.
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
By allowing the idle tokens to be re-selected in the following layers, IdleViT mitigates the negative impact of improper pruning in the early stages.
Plug n' Play: Channel Shuffle Module for Enhancing Tiny Vision Transformers
Inspired by the channel shuffle design in ShuffleNetV2 \cite{ma2018shufflenet}, our module expands the feature channels of a tiny ViT and partitions the channels into two groups: the \textit{Attended} and \textit{Idle} groups.
Improved Feature Distillation via Projector Ensemble
Motivated by the positive effect of the projector in feature distillation, we propose an ensemble of projectors to further improve the quality of student features.
