ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs

no code implementations • 6 Feb 2024 • Zhengyan Zhang, Yixin Song, Guanghui Yu, Xu Han, Yankai Lin, Chaojun Xiao, Chenyang Song, Zhiyuan Liu, Zeyu Mi, Maosong Sun

To find the most efficient activation function for sparse computation, we propose a systematic framework to examine the sparsity of LLMs from three aspects: the trade-off between sparsity and performance, the predictivity of sparsity, and the hardware affinity.

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

2 code implementations • 16 Dec 2023 • Yixin Song, Zeyu Mi, Haotong Xie, Haibo Chen

This paper introduces PowerInfer, a high-speed Large Language Model (LLM) inference engine on a personal computer (PC) equipped with a single consumer-grade GPU.

Language Modelling Large Language Model