1 code implementation • 16 Feb 2023 • Minghao Li, Ran Ben Basat, Shay Vargaftik, ChonLam Lao, Kevin Xu, Michael Mitzenmacher, Minlan Yu
To address this bottleneck and accelerate training, a widely-deployed approach is compression.
no code implementations • 17 Jan 2022 • Hao Wang, Yuxuan Qin, ChonLam Lao, Yanfang Le, Wenfei Wu, Kai Chen
However, switch memory is scarce compared to the volume of gradients transmitted in distributed training.