Search Results for author:
Found 4 papers, 3 papers with code
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning
This is because existing systems cannot handle workloads that include a mix of inference and PEFT finetuning requests.
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
Experiments show that QST can reduce the total memory footprint by up to 2. 3 $\times$ and speed up the finetuning process by up to 3 $\times$ while achieving competent performance compared with the state-of-the-art.
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) stand at the forefront, revolutionizing how we interact with our data.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
