Search Results for author:
Found 5 papers, 2 papers with code
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
FjORD alleviates the problem of client system heterogeneity by tailoring the model width to the client's capabilities.
Optimal Client Sampling for Federated Learning
We show that importance can be measured using only the norm of the update and give a formula for optimal client participation.
Natural Compression for Distributed Deep Learning
Our technique is applied individually to all entries of the to-be-compressed update vector and works by randomized rounding to the nearest (negative or positive) power of two, which can be computed in a "natural" way by ignoring the mantissa.
Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop
A key structural element in both of these methods is the inclusion of an outer loop at the beginning of which a full pass over the training data is made in order to compute the exact gradient, which is then used to construct a variance-reduced estimator of the gradient.
