Search Results for author:
Found 4 papers, 0 papers with code
Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
Large language models of high parameter counts are computationally expensive, yet can be made much more efficient by compressing their weights to very low numerical precision.
Scaling Laws for Post Training Quantized Large Language Models
Generalization abilities of well-trained large language models (LLMs) are known to scale predictably as a function of model size.
Post Training Quantization of Large Language Models with Microscaling Formats
Large Language Models (LLMs) have distinguished themselves with outstanding performance in complex language modeling tasks, yet they come with significant computational and storage challenges.
Self-Selected Attention Span for Accelerating Large Language Model Inference
We demonstrate that using this custom CUDA kernel improves the throughput of LLM inference by 28%.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
