Search Results for author:
Found 2 papers, 1 papers with code
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Compared to the baseline method, Mooncake can achieve up to a 525% increase in throughput in certain simulated scenarios while adhering to SLOs.
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
To enhance the efficiency of LLM decoding, we introduce model-attention disaggregation.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
