Search Results for author:
Found 8 papers, 5 papers with code
LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
The immense sizes of LLMs have led to very high resource demand and cost for running the models.
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Training LLMs at this scale brings unprecedented challenges to training efficiency and stability.
CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs
Considering the large space of DNN models and devices that impede direct profiling of all combinations, recent efforts focus on building a predictor to model the performance of DNN models on different devices.
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Distributed training using multiple devices (e. g., GPUs) has been widely adopted for learning DNN models over large datasets.
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
Extensive experiments on various GNN models and large graph datasets show that BGL significantly outperforms existing GNN training systems by 20. 68x on average.
DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters
DL2 is a DL-driven scheduler for DL clusters, targeting global training job expedition by dynamically resizing resources allocated to jobs.
Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters
A deep learning training job is resource-intensive and time-consuming.
Online Job Scheduling in Distributed Machine Learning Clusters
In a shared cluster handling multiple training jobs, a fundamental issue is how to efficiently schedule jobs and set the number of concurrent workers to run for each job, such that server resources are maximally utilized and model training can be completed in time.
Distributed, Parallel, and Cluster Computing
