Search Results for author:
Found 21 papers, 3 papers with code
HyperCLOVA X Technical Report
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding.
No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs).
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation
In this paper, we propose Dropping Backward Propagation (DropBP), a novel approach designed to reduce computational costs while maintaining accuracy.
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models
Weight-only quantization can be a promising approach, but sub-4 bit quantization remains a challenge due to large-magnitude activation outliers.
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models
To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task.
LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
Recent advances in self-supervised learning and the Transformer architecture have significantly improved natural language processing (NLP), achieving remarkably low perplexity.
Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression
Then, as an effort to push the compression ratio to the theoretical maximum (by entropy), we propose a sequential fixed-to-fixed encoding scheme.
Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
When the number of quantization bits is relatively low, however, non-convex optimization is unavoidable to improve model accuracy.
Modulating Regularization Frequency for Efficient Compression-Aware Model Training
While model compression is increasingly important because of large neural network size, compression-aware training is challenging as it needs sophisticated model modifications and longer training time. In this paper, we introduce regularization frequency (i. e., how often compression is performed during training) as a new regularization technique for a practical and efficient compression-aware training method.
Post-Training Weighted Quantization of Neural Networks for Language Models
As a practical model compression technique, parameter quantization is effective especially for language models associated with a large memory footprint.
Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation
Our analysis shows that for a given number of quantization bits, each block of Transformer contributes to translation quality and inference computations in different manners.
FleXOR: Trainable Fractional Quantization
Quantization based on the binary codes is gaining attention because each quantized bit can be directly utilized for computations without dequantization using look-up tables.
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs
Success of quantization in practice, hence, relies on an efficient computation engine design, especially for matrix multiplication that is a basic computation engine in most DNNs.
Decoupling Weight Regularization from Batch Size for Model Compression
Using various models, we show that simple weight updates to comply with compression formats along with long NR period is enough to achieve high compression ratio and model accuracy.
Network Pruning for Low-Rank Binary Index
In this paper, we propose a new network pruning technique that generates a low-rank binary index matrix to compress index data significantly.
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations.
Learning Low-Rank Approximation for CNNs
Low-rank approximation is an effective model compression technique to not only reduce parameter storage requirements, but to also reduce computations.
Network Pruning for Low-Rank Binary Indexing
Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs).
DeepTwist: Learning Model Compression via Occasional Weight Distortion
Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy.
Computation-Efficient Quantization Method for Deep Neural Networks
We present a non-intrusive quantization technique based on re-training the full precision model, followed by directly optimizing the corresponding binary model.
Retraining-Based Iterative Weight Quantization for Deep Neural Networks
We show that iterative retraining generates new sets of weights which can be quantized with decreasing quantization loss at each iteration.
