Distributed Optimization
Leaderboards
Greatest papers with code
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning.
L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework
Despite the importance of sparsity in many large-scale applications, there are few methods for distributed optimization of sparsity-inducing objectives.
Adding vs. Averaging in Distributed Primal-Dual Optimization
Distributed optimization methods for large-scale machine learning suffer from a communication bottleneck.
Federated Optimization in Heterogeneous Networks
Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the network (statistical heterogeneity).
Sparsified SGD with Memory
Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i. e. algorithms that leverage the compute power of many devices for training.
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
NeurIPS 2019
• epfml/powersgd
• 
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization.
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
NeurIPS 2019
• epfml/powersgd
•
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization.
Distributed Optimization with Arbitrary Local Solvers
To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally, and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods.
Robust Learning from Untrusted Sources
• NikolaKon1994/Robust-Learning-from-Untrusted-Sources
• 
Modern machine learning methods often require more data for training than a single expert can provide.
Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction
There is growing interest in large-scale machine learning and optimization over decentralized networks, e. g. in the context of multi-agent learning and federated learning.