Search Results for author:
Found 4 papers, 1 papers with code
BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching
Offline batch inference, which leverages the flexibility of request batching to achieve higher throughput and lower costs, is becoming more popular for latency-insensitive applications.
Disaggregating Embedding Recommendation Systems with FlexEMR
In this paper, we discuss the design of FlexEMR for optimized EMR disaggregation.
Symbolic Distillation for Learned TCP Congestion Control
At the core of our proposal is a novel symbolic branching algorithm that enables the rule to be aware of the context in terms of various network conditions, eventually converting the NN policy into a symbolic tree.
Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance
Today's auto-tuners (e. g., AutoTVM, Ansor) generate efficient tensor programs by navigating a large search space to identify effective implementations, but they do so with opaque hardware details.
