Automatic Configuration for Optimal Communication Scheduling in DNN Training

no code implementations • 27 Dec 2021 • Yiqing Ma, Hao Wang, Yiming Zhang, Kai Chen

ByteScheduler partitions and rearranges tensor transmissions to improve the communication efficiency of distributed Deep Neural Network (DNN) training.

Bayesian Optimization Scheduling