Search Results for author:
Found 10 papers, 1 papers with code
Towards Next-Level Post-Training Quantization of Hyper-Scale Transformers
Through extensive experiments on various language models and complexity analysis, we demonstrate that aespa is accurate and efficient in quantizing Transformer models.
Genie: Show Me the Data for Quantization
We also propose a post-training quantization algorithm to enhance the performance of quantized models.
Mr.BiQ: Post-Training Non-Uniform Quantization Based on Minimizing the Reconstruction Error
We thus propose a new post-training non-uniform quantization method, called Mr. BiQ, allowing low bit-width quantization even on Transformer models.
Modulating Regularization Frequency for Efficient Compression-Aware Model Training
While model compression is increasingly important because of large neural network size, compression-aware training is challenging as it needs sophisticated model modifications and longer training time. In this paper, we introduce regularization frequency (i. e., how often compression is performed during training) as a new regularization technique for a practical and efficient compression-aware training method.
Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
When the number of quantization bits is relatively low, however, non-convex optimization is unavoidable to improve model accuracy.
Post-Training Weighted Quantization of Neural Networks for Language Models
As a practical model compression technique, parameter quantization is effective especially for language models associated with a large memory footprint.
Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation
Our analysis shows that for a given number of quantization bits, each block of Transformer contributes to translation quality and inference computations in different manners.
FleXOR: Trainable Fractional Quantization
Quantization based on the binary codes is gaining attention because each quantized bit can be directly utilized for computations without dequantization using look-up tables.
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs
Success of quantization in practice, hence, relies on an efficient computation engine design, especially for matrix multiplication that is a basic computation engine in most DNNs.
Decoupling Weight Regularization from Batch Size for Model Compression
Using various models, we show that simple weight updates to comply with compression formats along with long NR period is enough to achieve high compression ratio and model accuracy.
