Search Results for author: Soummya Kar

Found 28 papers, 4 papers with code

A Fully Decentralized Tuning-free Inexact Projection Method for P2P Energy Trading

no code implementations12 Feb 2022 Meiyi Li, Javad Mohammadi, Soummya Kar

Agent-based solutions lend themselves well to address privacy concerns and the computational scalability needs of future distributed electric grids and end-use energy exchanges.

Decision Making

Variance reduced stochastic optimization over directed graphs with row and column stochastic weights

no code implementations7 Feb 2022 Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan

This paper proposes AB-SAGA, a first-order distributed stochastic optimization method to minimize a finite-sum of smooth and strongly convex functions distributed over an arbitrary directed graph.

Stochastic Optimization

Personalized Federated Learning via Convex Clustering

no code implementations1 Feb 2022 Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, Soummya Kar

The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized via a sum-of-norms penalty, weighted by a penalty parameter $\lambda$.

Personalized Federated Learning

Gradient Based Clustering

no code implementations1 Feb 2022 Aleksandar Armacki, Dragana Bajovic, Dusan Jakovetic, Soummya Kar

We propose a general approach for distance based clustering, using the gradient of the cost function that measures clustering quality with respect to cluster assignments and cluster center positions.

A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization

no code implementations12 Feb 2021 Ran Xin, Usman A. Khan, Soummya Kar

This paper considers decentralized stochastic optimization over a network of $n$ nodes, where each node possesses a smooth non-convex local cost function and the goal of the networked nodes is to find an $\epsilon$-accurate first-order stationary point of the sum of the local costs.

Stochastic Optimization

Learning to Solve AC Optimal Power Flow by Differentiating through Holomorphic Embeddings

no code implementations16 Dec 2020 Henning Lange, Bingqing Chen, Mario Berges, Soummya Kar

In this paper, we show efficient strategies that circumvent this problem by differentiating through the operations of a power flow solver that embeds the power flow equations into a holomorphic function.

A fast randomized incremental gradient method for decentralized non-convex optimization

no code implementations7 Nov 2020 Ran Xin, Usman A. Khan, Soummya Kar

For general smooth non-convex problems, we show the almost sure and mean-squared convergence of GT-SAGA to a first-order stationary point and further describe regimes of practical significance where it outperforms the existing approaches and achieves a network topology-independent iteration complexity respectively.

Fast decentralized non-convex finite-sum optimization with recursive variance reduction

no code implementations17 Aug 2020 Ran Xin, Usman A. Khan, Soummya Kar

We show that GT-SARAH, with appropriate algorithmic parameters, finds an $\epsilon$-accurate first-order stationary point with $O\big(\max\big\{N^{\frac{1}{2}}, n(1-\lambda)^{-2}, n^{\frac{2}{3}}m^{\frac{1}{3}}(1-\lambda)^{-1}\big\}L\epsilon^{-2}\big)$ gradient complexity, where ${(1-\lambda)\in(0, 1]}$ is the spectral gap of the network weight matrix and $L$ is the smoothness parameter of the cost functions.

Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphs

1 code implementation13 Aug 2020 Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan

In this paper, we propose Push-SAGA, a decentralized stochastic first-order method for finite-sum minimization over a directed network of nodes.

An improved convergence analysis for decentralized online stochastic non-convex optimization

no code implementations10 Aug 2020 Ran Xin, Usman A. Khan, Soummya Kar

In this paper, we study decentralized online stochastic non-convex optimization over a network of nodes.

S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs

2 code implementations15 May 2020 Muhammad I. Qureshi, Ran Xin, Soummya Kar, Usman A. Khan

In this report, we study decentralized stochastic optimization to minimize a sum of smooth and strongly convex cost functions when the functions are distributed over a directed network of nodes.

Stochastic Optimization

Distributed Gradient Methods for Nonconvex Optimization: Local and Global Convergence Guarantees

no code implementations23 Mar 2020 Brian Swenson, Soummya Kar, H. Vincent Poor, José M. F. Moura, Aaron Jaech

We discuss local minima convergence guarantees and explore the simple but critical role of the stable-manifold theorem in analyzing saddle-point avoidance.

Optimization and Control

Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima

no code implementations5 Mar 2020 Brian Swenson, Ryan Murray, Soummya Kar, H. Vincent Poor

In centralized settings, it is well known that stochastic gradient descent (SGD) avoids saddle points and converges to local minima in nonconvex problems.

Optimization and Control

Gradient tracking and variance reduction for decentralized optimization and machine learning

no code implementations13 Feb 2020 Ran Xin, Soummya Kar, Usman A. Khan

Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data is distributed over a network of nodes and raw data sharing is not permitted due to privacy and/or resource constraints.

Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG

no code implementations8 Oct 2019 Ran Xin, Usman A. Khan, Soummya Kar

Decentralized stochastic optimization has recently benefited from gradient tracking methods \cite{DSGT_Pu, DSGT_Xin} providing efficient solutions for large-scale empirical risk minimization problems.

Stochastic Optimization

Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking

no code implementations25 Sep 2019 Ran Xin, Usman A. Khan, Soummya Kar

In this paper, we study decentralized empirical risk minimization problems, where the goal to minimize a finite-sum of smooth and strongly-convex functions available over a network of nodes.

Optimization and Control

An introduction to decentralized stochastic optimization with gradient tracking

no code implementations23 Jul 2019 Ran Xin, Soummya Kar, Usman A. Khan

Decentralized solutions to finite-sum minimization are of significant importance in many signal processing, control, and machine learning applications.

Stochastic Optimization

MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling

3 code implementations23 May 2019 Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang, Gauri Joshi, Soummya Kar

This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network.

Distributed stochastic optimization with gradient tracking over strongly-connected networks

no code implementations18 Mar 2019 Ran Xin, Anit Kumar Sahu, Usman A. Khan, Soummya Kar

In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph.

Stochastic Optimization

Clustering with Distributed Data

no code implementations1 Jan 2019 Soummya Kar, Brian Swenson

By appropriate choice of $\rho$, the set of generalized minima may be brought arbitrarily close to the set of Lloyd's minima.

Towards Gradient Free and Projection Free Stochastic Optimization

no code implementations8 Oct 2018 Anit Kumar Sahu, Manzil Zaheer, Soummya Kar

This paper focuses on the problem of \emph{constrained} \emph{stochastic} optimization.

Stochastic Optimization

Coded Distributed Computing for Inverse Problems

no code implementations NeurIPS 2017 Yaoqing Yang, Pulkit Grover, Soummya Kar

Our experiments for personalized PageRank performed on real systems and real social networks show that this ratio can be as large as $10^4$.

Distributed Computing

Topology Adaptive Graph Convolutional Networks

2 code implementations ICLR 2018 Jian Du, Shanghang Zhang, Guanhang Wu, Jose M. F. Moura, Soummya Kar

Spectral graph convolutional neural networks (CNNs) require approximation to the convolution to alleviate the computational complexity, resulting in performance loss.

Convergence analysis of belief propagation for pairwise linear Gaussian models

no code implementations12 Jun 2017 Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura

Gaussian belief propagation (BP) has been widely used for distributed inference in large-scale networks such as the smart grid, sensor networks, and social networks, where local measurements/observations are scattered over a wide geographical area.

Convergence analysis of the information matrix in Gaussian belief propagation

no code implementations13 Apr 2017 Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura

Gaussian belief propagation (BP) has been widely used for distributed estimation in large-scale networks such as the smart grid, communication networks, and social networks, where local measurements/observations are scattered over a wide geographical area.

Convergence Analysis of Distributed Inference with Vector-Valued Gaussian Belief Propagation

no code implementations7 Nov 2016 Jian Du, Shaodan Ma, Yik-Chung Wu, Soummya Kar, José M. F. Moura

A necessary and sufficient convergence condition for the belief mean vector to converge to the optimal centralized estimator is provided under the assumption that the message information matrix is initialized as a positive semidefinite matrix.

Cannot find the paper you are looking for? You can Submit a new open access paper.