Search Results for author:
Found 1 papers, 0 papers with code
Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
However, existing model serving systems cannot achieve adequate batch sizes while meeting latency objectives as these systems eagerly dispatch requests to accelerators to minimize the accelerator idle time.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
