Search Results for author:
Found 8 papers, 2 papers with code
Federated Online and Bandit Convex Optimization
Furthermore, we delve into the more challenging setting of federated online optimization with bandit (zeroth-order) feedback, where the machines can only access values of the cost functions at the queried points.
On the Effect of Defections in Federated Learning and How to Prevent Them
Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model.
A Stochastic Newton Algorithm for Distributed Convex Optimization
We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same population objective, as well as stochastic Hessian-vector products (products of an independent unbiased estimator of the Hessian of the population objective with arbitrary vectors), with many such stochastic computations performed between rounds of communication.
Minibatch vs Local SGD for Heterogeneous Distributed Learning
the average objective; and machines can only communicate intermittently.
Is Local SGD Better than Minibatch SGD?
We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method.
Communication trade-offs for Local-SGD with large step size
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
Communication trade-offs for synchronized distributed SGD with large step size
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
Don't Use Large Mini-Batches, Use Local SGD
Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks.
