ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models

1 code implementation • 21 Feb 2024 • Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun

Some recent efforts have explored introducing ReLU or its variants as the substitutive activation function to help LLMs achieve activation sparsity and inference acceleration, but few can simultaneously obtain high sparsity and comparable model performance.

Scalar Product Lattice Computation for Efficient Privacy-preserving Systems

1 code implementation • 4 Apr 2020 • Yogachandran Rahulamathavan, Safak Dogan, Xiyu Shi, Rongxing Lu, Muttukrishnan Rajarajan, Ahmet Kondoz

Privacy-preserving applications allow users to perform on-line daily actions without leaking sensitive information.

Cryptography and Security