Achieving the fundamental convergence-communication tradeoff with Differentially Quantized Gradient Descent

6 Feb 2020Chung-Yi LinVictoria KostinaBabak Hassibi

The problem of reducing the communication cost in distributed training through gradient quantization is considered. For gradient descent on smooth and strongly convex objective functions on $\mathbb{R}^n$, we characterize the fundamental rate function-the minimum achievable linear convergence rate for a given number of bits per dimension $n$... (read more)

PDF Abstract


No code implementations yet. Submit your code now

Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper

🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet