Search Results for author:
Found 2 papers, 0 papers with code
KunServe: Elastic and Efficient Large Language Model Serving with Parameter-centric Memory Management
The stateful nature of large language model (LLM) servingcan easily throttle precious GPU memory under load burstor long-generation requests like chain-of-thought reasoning, causing latency spikes due to queuing incoming requests.
Characterizing the Dilemma of Performance and Index Size in Billion-Scale Vector Search and Breaking It with Second-Tier Memory
In this paper, we are the first to characterize the trade-off of performance and index size in existing SSD-based graph and cluster indexes: to improve throughput by 5. 7$\times$ and 1. 7$\times$, these indexes have to pay a 5. 8$\times$ storage amplification and 7. 7$\times$ with respect to the dataset size, respectively.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
