Communication optimization strategies for distributed deep neural network training: A survey

no code implementations • 6 Mar 2020 • Shuo Ouyang, Dezun Dong, Yemao Xu, Liquan Xiao

At the algorithm level, we describe how to reduce the number of communication rounds and transmitted bits per round.