Search Results for author:
Found 5 papers, 0 papers with code
Faster Inference of Integer SWIN Transformer by Removing the GELU Activation
In this work, we improve upon the inference latency of the state-of-the-art methods by removing the floating-point operations, which are associated with the GELU activation in Swin Transformer.
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation
Previously researchers were focused on lower bit-width integer data types for the forward propagation of language models to save memory and computation.
Efficient Fine-Tuning of Compressed Language Models with Learners
We introduce Learner modules and priming, novel methods for fine-tuning that exploit the overparameterization of pre-trained language models to gain benefits in convergence speed and resource utilization.
Is Integer Arithmetic Enough for Deep Learning Training?
As such, quantization has attracted the attention of researchers in recent years.
Efficient Fine-Tuning of BERT Models on the Edge
FAR reduces fine-tuning time on the DistilBERT model and CoLA dataset by 30%, and time spent on memory operations by 47%.
