Search Results for author:
Found 5 papers, 4 papers with code
A Simple Linear Patch Revives Layer-Pruned Large Language Models
With only 5K samples, the retained performance of LinearPatch can be further boosted to 95. 16% within 30 minutes on a single computing card.
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
In addition, strategically scaling the model sizes or reasoning steps can effectively enhance the performance.
FlatQuant: Flatness Matters for LLM Quantization
In this paper, we propose FlatQuant (Fast and Learnable Affine Transformation), a new post-training quantization approach to enhance flatness of weights and activations.
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
Such outliers are found to allocate most of the attention scores on initial tokens of input, termed as pivot tokens, which are crucial to the performance of quantized LLMs.
Learning Imbalanced Data with Vision Transformers
In this paper, we systematically investigate the ViTs' performance in LTR and propose LiVT to train ViTs from scratch only with LT data.
Ranked #7 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
