Search Results for author: Samuel Horvath

Found 5 papers, 2 papers with code

Optimal Client Sampling for Federated Learning

1 code implementation NeurIPS 2021 Wenlin Chen, Samuel Horvath, Peter Richtarik

We show that importance can be measured using only the norm of the update and give a formula for optimal client participation.

Federated Learning

Natural Compression for Distributed Deep Learning

no code implementations27 May 2019 Samuel Horvath, Chen-Yu Ho, Ludovit Horvath, Atal Narayan Sahu, Marco Canini, Peter Richtarik

Our technique is applied individually to all entries of the to-be-compressed update vector and works by randomized rounding to the nearest (negative or positive) power of two, which can be computed in a "natural" way by ignoring the mantissa.

Quantization

Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop

no code implementations24 Jan 2019 Dmitry Kovalev, Samuel Horvath, Peter Richtarik

A key structural element in both of these methods is the inclusion of an outer loop at the beginning of which a full pass over the training data is made in order to compute the exact gradient, which is then used to construct a variance-reduced estimator of the gradient.

Cannot find the paper you are looking for? You can Submit a new open access paper.