Search Results for author:
Found 3 papers, 0 papers with code
SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
Early exiting has recently emerged as a promising technique for accelerating large language models (LLMs) by effectively reducing the hardware computation and memory access.
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
We compare the performance of the same optimization methods across different hardware platforms, the performance across different hardware platforms, and the performance of different methods on the same hardware platform.
Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantization
To tackle these challenges and enable fast and efficient LLM inference on GPUs, we propose the following techniques in this paper.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
