Multi-model Machine Learning Inference Serving with GPU Spatial Partitioning

no code implementations • 1 Sep 2021 • Seungbeom Choi, Sunho Lee, Yeonjae Kim, Jongse Park, Youngjin Kwon, Jaehyuk Huh

To maximize the resource efficiency of inference servers, a key mechanism proposed in this paper is to exploit hardware support for spatial partitioning of GPU resources.

BIG-bench Machine Learning Scheduling