Search Results for author:
Found 3 papers, 0 papers with code
FlashOverlap: A Lightweight Design for Efficiently Overlapping Communication and Computation
We identify that an efficient and adaptable overlapping design should satisfy (1) tile-wise overlapping to maximize the overlapping opportunity, (2) interference-free computation to maintain the original computational performance, and (3) communication agnosticism to reduce the development burden against varying communication primitives.
semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
In this paper, we identify that the advantage of the disaggregated system lies in the disaggregated computation, i. e., partitioning the computational resource to enable the asynchronous computation of two phases.
FlashDecoding++: Faster Large Language Model Inference on GPUs
A single and static dataflow may lead to a 50. 25% performance loss for GEMMs of different shapes in LLM inference.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
