This section contains a compilation of distributed communication methods for scaling deep learning to very large models. Communication methods aim to minimize communication overhead between nodes in a distributed system, and link nodes in an optimal way.