Search Results for author: Rafał Szlendak

Found 2 papers, 0 papers with code

Understanding Progressive Training Through the Framework of Randomized Coordinate Descent

no code implementations6 Jun 2023 Rafał Szlendak, Elnur Gasanov, Peter Richtárik

We propose a Randomized Progressive Training algorithm (RPT) -- a stochastic proxy for the well-known Progressive Training method (PT) (Karras et al., 2017).

Permutation Compressors for Provably Faster Distributed Nonconvex Optimization

no code implementations ICLR 2022 Rafał Szlendak, Alexander Tyurin, Peter Richtárik

In this paper we i) extend the theory of MARINA to support a much wider class of potentially {\em correlated} compressors, extending the reach of the method beyond the classical independent compressors setting, ii) show that a new quantity, for which we coin the name {\em Hessian variance}, allows us to significantly refine the original analysis of MARINA without any additional assumptions, and iii) identify a special class of correlated compressors based on the idea of {\em random permutations}, for which we coin the term Perm$K$, the use of which leads to $O(\sqrt{n})$ (resp.

Cannot find the paper you are looking for? You can Submit a new open access paper.