Search Results for author:
Found 9 papers, 4 papers with code
AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
To substantiate the efficacy of our method, we experimentally show, how the introduction of adaptive step size and adaptive batch size gradually improves the performance of regular SGD.
SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
Adaptive optimization methods are widely recognized as among the most popular approaches for training Deep Neural Networks (DNNs).
Stochastic Gradient Descent with Preconditioned Polyak Step-size
Stochastic Gradient Descent (SGD) is one of the many iterative optimization methods that are widely used in solving machine learning problems.
Suppressing Poisoning Attacks on Federated Learning for Medical Imaging
Federated Learning (FL) is a promising solution that enables collaborative training through exchange of model parameters instead of raw data.
Stochastic Gradient Methods with Preconditioned Updates
There are several algorithms for such problems, but existing methods often work poorly when the problem is badly scaled and/or ill-conditioned, and a primary goal of this work is to introduce methods that alleviate this issue.
Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimization
Statistical preconditioning enables fast methods for distributed large-scale empirical risk minimization problems.
Distributed Optimization
Optimization and Control
Inexact Tensor Methods and Their Application to Stochastic Convex Optimization
We propose general non-accelerated and accelerated tensor methods under inexact information on the derivatives of the objective, analyze their convergence rate.
Optimization and Control
Recent Theoretical Advances in Non-Convex Optimization
For this setting, we first present known results for the convergence rates of deterministic first-order methods, which are then followed by a general theoretical analysis of optimal stochastic and randomized gradient schemes, and an overview of the stochastic first-order methods.
Accelerated meta-algorithm for convex optimization
We propose an accelerated meta-algorithm, which allows to obtain accelerated methods for convex unconstrained minimization in different settings.
Optimization and Control
