This section contains a compilation of distributed hybrid parallel methods for scaling deep learning to very large models. Hybrid methods combine data parallel and model parallel methods.

Subcategories

• 1 2D Parallel Distributed Methods

• 2 Parameter Server Methods