no code implementations • 7 Nov 2016 • Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura
A necessary and sufficient convergence condition for the belief mean vector to converge to the optimal centralized estimator is provided under the assumption that the message information matrix is initialized as a positive semidefinite matrix.
no code implementations • 13 Apr 2017 • Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura
Gaussian belief propagation (BP) has been widely used for distributed estimation in large-scale networks such as the smart grid, communication networks, and social networks, where local measurements/observations are scattered over a wide geographical area.
no code implementations • 12 Jun 2017 • Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura
Gaussian belief propagation (BP) has been widely used for distributed inference in large-scale networks such as the smart grid, sensor networks, and social networks, where local measurements/observations are scattered over a wide geographical area.
2 code implementations • ICLR 2018 • Jian Du, Shanghang Zhang, Guanhang Wu, Jose M. F. Moura, Soummya Kar
Spectral graph convolutional neural networks (CNNs) require approximation to the convolution to alleviate the computational complexity, resulting in performance loss.
no code implementations • NeurIPS 2017 • Yaoqing Yang, Pulkit Grover, Soummya Kar
Our experiments for personalized PageRank performed on real systems and real social networks show that this ratio can be as large as $10^4$.
no code implementations • 8 Oct 2018 • Anit Kumar Sahu, Manzil Zaheer, Soummya Kar
This paper focuses on the problem of \emph{constrained} \emph{stochastic} optimization.
no code implementations • 1 Jan 2019 • Soummya Kar, Brian Swenson
By appropriate choice of $\rho$, the set of generalized minima may be brought arbitrarily close to the set of Lloyd's minima.
no code implementations • 18 Mar 2019 • Ran Xin, Anit Kumar Sahu, Usman A. Khan, Soummya Kar
In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph.
no code implementations • 18 Mar 2019 • Brian Swenson, Soummya Kar, H. Vincent Poor, Jose' M. F. Moura
The paper considers algorithms for optimizing the sum function.
4 code implementations • 23 May 2019 • Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang, Gauri Joshi, Soummya Kar
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network.
no code implementations • 23 Jul 2019 • Ran Xin, Soummya Kar, Usman A. Khan
Decentralized solutions to finite-sum minimization are of significant importance in many signal processing, control, and machine learning applications.
no code implementations • 25 Sep 2019 • Ran Xin, Usman A. Khan, Soummya Kar
In this paper, we study decentralized empirical risk minimization problems, where the goal to minimize a finite-sum of smooth and strongly-convex functions available over a network of nodes.
Optimization and Control
no code implementations • 8 Oct 2019 • Ran Xin, Usman A. Khan, Soummya Kar
Decentralized stochastic optimization has recently benefited from gradient tracking methods \cite{DSGT_Pu, DSGT_Xin} providing efficient solutions for large-scale empirical risk minimization problems.
no code implementations • 13 Feb 2020 • Ran Xin, Soummya Kar, Usman A. Khan
Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data is distributed over a network of nodes and raw data sharing is not permitted due to privacy and/or resource constraints.
no code implementations • 5 Mar 2020 • Brian Swenson, Ryan Murray, Soummya Kar, H. Vincent Poor
In centralized settings, it is well known that stochastic gradient descent (SGD) avoids saddle points and converges to local minima in nonconvex problems.
Optimization and Control
no code implementations • 23 Mar 2020 • Brian Swenson, Soummya Kar, H. Vincent Poor, José M. F. Moura, Aaron Jaech
We discuss local minima convergence guarantees and explore the simple but critical role of the stable-manifold theorem in analyzing saddle-point avoidance.
Optimization and Control
2 code implementations • 15 May 2020 • Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan
In this report, we study decentralized stochastic optimization to minimize a sum of smooth and strongly convex cost functions when the functions are distributed over a directed network of nodes.
no code implementations • 10 Aug 2020 • Ran Xin, Usman A. Khan, Soummya Kar
In this paper, we study decentralized online stochastic non-convex optimization over a network of nodes.
1 code implementation • 13 Aug 2020 • Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan
In this paper, we propose Push-SAGA, a decentralized stochastic first-order method for finite-sum minimization over a directed network of nodes.
no code implementations • 17 Aug 2020 • Ran Xin, Usman A. Khan, Soummya Kar
We show that GT-SARAH, with appropriate algorithmic parameters, finds an $\epsilon$-accurate first-order stationary point with $O\big(\max\big\{N^{\frac{1}{2}}, n(1-\lambda)^{-2}, n^{\frac{2}{3}}m^{\frac{1}{3}}(1-\lambda)^{-1}\big\}L\epsilon^{-2}\big)$ gradient complexity, where ${(1-\lambda)\in(0, 1]}$ is the spectral gap of the network weight matrix and $L$ is the smoothness parameter of the cost functions.
no code implementations • 7 Nov 2020 • Ran Xin, Usman A. Khan, Soummya Kar
For general smooth non-convex problems, we show the almost sure and mean-squared convergence of GT-SAGA to a first-order stationary point and further describe regimes of practical significance where it outperforms the existing approaches and achieves a network topology-independent iteration complexity respectively.
no code implementations • 16 Dec 2020 • Henning Lange, Bingqing Chen, Mario Berges, Soummya Kar
In this paper, we show efficient strategies that circumvent this problem by differentiating through the operations of a power flow solver that embeds the power flow equations into a holomorphic function.
no code implementations • 12 Feb 2021 • Ran Xin, Usman A. Khan, Soummya Kar
This paper considers decentralized stochastic optimization over a network of $n$ nodes, where each node possesses a smooth non-convex local cost function and the goal of the networked nodes is to find an $\epsilon$-accurate first-order stationary point of the sum of the local costs.
no code implementations • 1 Feb 2022 • Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, Soummya Kar
We propose a general approach for distance based clustering, using the gradient of the cost function that measures clustering quality with respect to cluster assignments and cluster center positions.
no code implementations • 1 Feb 2022 • Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, Soummya Kar
The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized via a sum-of-norms penalty, weighted by a penalty parameter $\lambda$.
no code implementations • 7 Feb 2022 • Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan
This paper proposes AB-SAGA, a first-order distributed stochastic optimization method to minimize a finite-sum of smooth and strongly convex functions distributed over an arbitrary directed graph.
no code implementations • 12 Feb 2022 • Meiyi Li, Javad Mohammadi, Soummya Kar
Agent-based solutions lend themselves well to address privacy concerns and the computational scalability needs of future distributed electric grids and end-use energy exchanges.
no code implementations • 6 Apr 2022 • Dusan Jakovetic, Dragana Bajovic, Anit Kumar Sahu, Soummya Kar, Nemanja Milosevic, Dusan Stamenkovic
We introduce a general framework for nonlinear stochastic gradient descent (SGD) for the scenarios when gradient noise exhibits heavy tails.
no code implementations • 11 Jul 2022 • Carmel Fiscko, Soummya Kar, Bruno Sinopoli
To efficiently find a policy in this rapidly scaling space, we propose a clustered Bellman operator that optimizes over the action space for one cluster at any evaluation.
no code implementations • 11 Jul 2022 • Meiyi Li, Yuhan Du, Javad Mohammadi, Constance Crozier, Kyri Baker, Soummya Kar
Linear approximations of the AC power flow equations are of great significance for the computational efficiency of large-scale optimal power flow (OPF) problems.
no code implementations • 22 Sep 2022 • Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, Soummya Kar
In the proposed setup, the grouping of users (based on the data distributions they sample), as well as the underlying statistical properties of the distributions, are apriori unknown.
no code implementations • 25 Oct 2022 • Stefan Vlaski, Soummya Kar, Ali H. Sayed, José M. F. Moura
Moreover, and significantly, theory and applications show that networked agents, through cooperation and sharing, are able to match the performance of cloud or federated solutions, while offering the potential for improved privacy, increasing resilience, and saving resources.
no code implementations • 2 Nov 2022 • Dragana Bajovic, Dusan Jakovetic, Soummya Kar
In this work we provide a formal framework for the study of general high probability bounds with SGD, based on the theory of large deviations.
no code implementations • 15 Nov 2022 • Aayushya Agarwal, Carmel Fiscko, Soummya Kar, Larry Pileggi, Bruno Sinopoli
To find the value of the critical point, we propose a time step search routine for Forward Euler discretization that controls the local truncation error, a method adapted from circuit simulation ideas.
no code implementations • 5 Feb 2023 • Carmel Fiscko, Soummya Kar, Bruno Sinopoli
In this work we investigate an importance sampling approach for evaluating policies for a structurally time-varying factored Markov decision process (MDP), i. e. the policy's value is estimated with a high-probability confidence interval.
no code implementations • 24 Apr 2023 • Carmel Fiscko, Soummya Kar, Bruno Sinopoli
The controller's objective is to find an optimal policy that maximizes the value of the expected system given a priori knowledge of the agents' dropout probabilities.
no code implementations • 21 Oct 2023 • Carmel Fiscko, Aayushya Agarwal, Yihan Ruan, Soummya Kar, Larry Pileggi, Bruno Sinopoli
We present a stochastic first-order optimization method specialized for deep neural networks (DNNs), ECCO-DNN.
no code implementations • 28 Oct 2023 • Aleksandar Armacki, Pranay Sharma, Gauri Joshi, Dragana Bajovic, Dusan Jakovetic, Soummya Kar
We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise.
no code implementations • 2 Feb 2024 • Aleksandar Armacki, Dragana Bajović, Dušan Jakovetić, Soummya Kar
The proposed family, termed Distributed Gradient Clustering (DGC-$\mathcal{F}_\rho$), is parametrized by $\rho \geq 1$, controling the proximity of users' center estimates, with $\mathcal{F}$ determining the clustering loss.
no code implementations • 9 Mar 2024 • Ritabrata Ray, Yorie Nakahira, Soummya Kar
In this paper, we study the problem of ensuring safety with a few shots of samples for partially unknown systems.
no code implementations • 12 Apr 2024 • Cláudio Gomes, João Paulo Fernandes, Gabriel Falcao, Soummya Kar, Sridhar Tayur
The rapid adoption of Electric Vehicles (EVs) poses challenges for electricity grids to accommodate or mitigate peak demand.