Search Results for author:
Found 4 papers, 0 papers with code
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
In this work, we observed the saturation of computationally expensive feed-forward blocks of LLM layers and proposed FFN-SkipLLM, which is a novel fine-grained skip strategy of autoregressive LLMs.
Mr.BiQ: Post-Training Non-Uniform Quantization Based on Minimizing the Reconstruction Error
We thus propose a new post-training non-uniform quantization method, called Mr. BiQ, allowing low bit-width quantization even on Transformer models.
Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
When the number of quantization bits is relatively low, however, non-convex optimization is unavoidable to improve model accuracy.
Post-Training Weighted Quantization of Neural Networks for Language Models
As a practical model compression technique, parameter quantization is effective especially for language models associated with a large memory footprint.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
