Search Results for author:
Found 6 papers, 4 papers with code
SaiT: Sparse Vision Transformers through Adaptive Token Pruning
Sparse adaptive image Transformer (SaiT) offers varying levels of model acceleration by merely changing the token sparsity on the fly.
MaiT: Leverage Attention Masks for More Efficient Image Transformers
Though image transformers have shown competitive results with convolutional neural networks in computer vision tasks, lacking inductive biases such as locality still poses problems in terms of model efficiency especially for embedded applications.
A Fast Post-Training Pruning Framework for Transformers
To address this, we propose a fast post-training pruning framework for Transformers that does not require any retraining.
Fast and Efficient Once-For-All Networks for Diverse Hardware Deployment
In this work, we continue the effort to reduce the training cost of OFA methods.
Learned Token Pruning for Transformers
We extensively test the performance of LTP on GLUE tasks and show that our method outperforms the prior state-of-the-art token pruning methods by up to ~2. 5% higher accuracy with the same amount of FLOPs.
Post-Training Piecewise Linear Quantization for Deep Neural Networks
Quantization plays an important role in the energy-efficient deployment of deep neural networks on resource-limited devices.
