Search Results for author:
Found 14 papers, 2 papers with code
Variance reduced stochastic optimization over directed graphs with row and column stochastic weights
This paper proposes AB-SAGA, a first-order distributed stochastic optimization method to minimize a finite-sum of smooth and strongly convex functions distributed over an arbitrary directed graph.
A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization
This paper considers decentralized stochastic optimization over a network of $n$ nodes, where each node possesses a smooth non-convex local cost function and the goal of the networked nodes is to find an $\epsilon$-accurate first-order stationary point of the sum of the local costs.
A fast randomized incremental gradient method for decentralized non-convex optimization
For general smooth non-convex problems, we show the almost sure and mean-squared convergence of GT-SAGA to a first-order stationary point and further describe regimes of practical significance where it outperforms the existing approaches and achieves a network topology-independent iteration complexity respectively.
A general framework for decentralized optimization with first-order methods
Decentralized optimization to minimize a finite sum of functions over a network of nodes has been a significant focus within control and signal processing research due to its natural relevance to optimal control and signal estimation problems.
Fast decentralized non-convex finite-sum optimization with recursive variance reduction
We show that GT-SARAH, with appropriate algorithmic parameters, finds an $\epsilon$-accurate first-order stationary point with $O\big(\max\big\{N^{\frac{1}{2}}, n(1-\lambda)^{-2}, n^{\frac{2}{3}}m^{\frac{1}{3}}(1-\lambda)^{-1}\big\}L\epsilon^{-2}\big)$ gradient complexity, where ${(1-\lambda)\in(0, 1]}$ is the spectral gap of the network weight matrix and $L$ is the smoothness parameter of the cost functions.
Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphs
In this paper, we propose Push-SAGA, a decentralized stochastic first-order method for finite-sum minimization over a directed network of nodes.
An improved convergence analysis for decentralized online stochastic non-convex optimization
In this paper, we study decentralized online stochastic non-convex optimization over a network of nodes.
S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs
In this report, we study decentralized stochastic optimization to minimize a sum of smooth and strongly convex cost functions when the functions are distributed over a directed network of nodes.
Gradient tracking and variance reduction for decentralized optimization and machine learning
Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data is distributed over a network of nodes and raw data sharing is not permitted due to privacy and/or resource constraints.
Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG
Decentralized stochastic optimization has recently benefited from gradient tracking methods \cite{DSGT_Pu, DSGT_Xin} providing efficient solutions for large-scale empirical risk minimization problems.
Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking
In this paper, we study decentralized empirical risk minimization problems, where the goal to minimize a finite-sum of smooth and strongly-convex functions available over a network of nodes.
Optimization and Control
An introduction to decentralized stochastic optimization with gradient tracking
Decentralized solutions to finite-sum minimization are of significant importance in many signal processing, control, and machine learning applications.
Distributed stochastic optimization with gradient tracking over strongly-connected networks
In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph.
Distributed Nesterov gradient methods over arbitrary graphs
In this letter, we introduce a distributed Nesterov method, termed as $\mathcal{ABN}$, that does not require doubly-stochastic weight matrices.
