Search Results for author:
Found 5 papers, 2 papers with code
A Frequency-aware Software Cache for Large Recommendation System Embeddings
Deep learning recommendation models (DLRMs) have been widely applied in Internet companies.
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
The success of Transformer models has pushed the deep learning model scale to billions of parameters.
Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters
ONES automatically manages the elasticity of each job based on the training batch size, so as to maximize GPU utilization and improve scheduling efficiency.
Tesseract: Parallelize the Tensor Parallelism Efficiently
It increases efficiency by reducing communication overhead and lowers the memory required for each GPU.
Maximizing Parallelism in Distributed Training for Huge Neural Networks
Our work is the first to introduce a 3-dimensional model parallelism for expediting huge language models.
