Search Results for author: Yubin Shi

Found 3 papers, 0 papers with code

ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization

no code implementations12 Nov 2024 Weibo Zhao, Yubin Shi, Xinyu Lyu, Wanchen Sui, Shen Li, Yong Li

Quantization stands as a pivotal technique for large language model (LLM) serving, yet it poses significant challenges particularly in achieving effective low-bit quantization.

Language Modeling Language Modelling +3

Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models

no code implementations NeurIPS 2023 Yubin Shi, Yixuan Chen, Mingzhi Dong, Xiaochen Yang, Dongsheng Li, Yujiang Wang, Robert P. Dick, Qin Lv, Yingying Zhao, Fan Yang, Tun Lu, Ning Gu, Li Shang

To describe such modular-level learning capabilities, we introduce a novel concept dubbed modular neural tangent kernel (mNTK), and we demonstrate that the quality of a module's learning is tightly associated with its mNTK's principal eigenvalue $\lambda_{\max}$.

Cannot find the paper you are looking for? You can Submit a new open access paper.