no code implementations • 2 Nov 2022 • Haoze He, Parijat Dube
In this paper, we propose the (de)centralized Non-blocking SGD (Non-blocking SGD) which can address the straggler problem in a heterogeneous environment.
no code implementations • 2 Nov 2022 • Haoze He, Parijat Dube
The convergence of SGD based distributed training algorithms is tied to the data distribution across workers.