Search Results for author:
Found 4 papers, 0 papers with code
EControl: Fast Distributed Optimization with Compression and Error Control
Error Compensation (EC) is an extremely popular mechanism to mitigate the aforementioned issues during the training of models enhanced by contractive compression operators.
Special Properties of Gradient Descent with Large Learning Rates
However, we show through a novel set of experiments that the stochastic noise is not sufficient to explain good non-convex training, and that instead the effect of a large learning rate itself is essential for obtaining best performance. We demonstrate the same effects also in the noise-less case, i. e. for full-batch GD.
ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!
The canonical approach to solving such problems is via the proximal gradient descent (ProxGD) algorithm, which is based on the evaluation of the gradient of $f$ and the prox operator of $\psi$ in each iteration.
Stochastic continuum armed bandit problem of few linear parameters in high dimensions
We consider a stochastic continuum armed bandit problem where the arms are indexed by the $\ell_2$ ball $B_{d}(1+\nu)$ of radius $1+\nu$ in $\mathbb{R}^d$.
