Search Results for author:
Found 6 papers, 3 papers with code
MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
This paper presents MoE-Gen, a high-throughput MoE inference system optimized for single-GPU execution.
MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
Its key innovation is a sparsity-aware CAP analysis model, the first to integrate cost, performance, and accuracy metrics into a single diagram while estimating the impact of sparsity on system performance.
MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
This paper presents MoE-Infinity, an efficient MoE inference system designed for personal machines with limited GPU memory capacity.
ServerlessLLM: Low-Latency Serverless Inference for Large Language Models
This paper presents ServerlessLLM, a distributed system designed to support low-latency serverless inference for Large Language Models (LLMs).
Enhancing the long-term performance of recommender system
Notably, an optimal parameter n* of ARL existed in long-term recommendation, indicating that there is a trade-off between keeping diversity of item and user's preference to maximize the long-term recommendation accuracy.
Predictability of diffusion-based recommender systems
The results show that the higher recommendation accuracy based on diffusion algorithms can still be achieved by optimizing the way of resource allocation on a density network.
