Search Results for author:
Found 2 papers, 1 papers with code
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
