Search Results for author:
Found 6 papers, 6 papers with code
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Additionally, an adaptive strategy is designed to autonomously determine the optimal number of sub-channels for channel disassembly.
Lossy and Lossless (L$^2$) Post-training Model Size Compression
Then, a dedicated differentiable counter is introduced to guide the optimization of lossy compression to arrive at a more suitable point for later lossless compression.
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
The channel-wise shifting aligns the center of each channel for removal of outlier asymmetry.
Lossy and Lossless (L2) Post-training Model Size Compression
Then, a dedicated differentiable counter is introduced to guide the optimization of lossy compression to arrive at a more suitable point for later lossless compression.
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models
With the trends of large NLP models, the increasing memory and computation costs hinder their efficient deployment on resource-limited devices.
QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization
With QDROP, the limit of PTQ is pushed to the 2-bit activation for the first time and the accuracy boost can be up to 51. 49%.
