LUT Tensor Core: Lookup Table Enables Efficient Low-Bit LLM Inference Acceleration

no code implementations • 12 Aug 2024 • Zhiwen Mo, Lei Wang, Jianyu Wei, Zhichen Zeng, Shijie Cao, Lingxiao Ma, Naifeng Jing, Ting Cao, Jilong Xue, Fan Yang, Mao Yang

Then, LUT Tensor Core proposes the hardware design featuring an elongated tiling shape design to enhance table reuse and a bit-serial design to support various precision combinations in mpGEMM.

Language Modelling Large Language Model