Search Results for author: Pranay Sharma

Found 16 papers, 2 papers with code

FedAST: Federated Asynchronous Simultaneous Training

no code implementations1 Jun 2024 Baris Askin, Pranay Sharma, Carlee Joe-Wong, Gauri Joshi

Much of the existing work in FL focuses on efficiently learning a model for a single task.

Federated Learning

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise

no code implementations28 Oct 2023 Aleksandar Armacki, Pranay Sharma, Gauri Joshi, Dragana Bajovic, Dusan Jakovetic, Soummya Kar

First, for non-convex costs and component-wise nonlinearities, we establish a convergence rate arbitrarily close to $\mathcal{O}\left(t^{-\frac{1}{4}}\right)$, whose exponent is independent of noise and problem parameters.

Quantization

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation

1 code implementation2 Jun 2023 Davoud Ataee Tarzanagh, Mingchen Li, Pranay Sharma, Samet Oymak

Stochastic approximation with multiple coupled sequences (MSA) has found broad applications in machine learning as it encompasses a rich class of problems including bilevel optimization (BLO), multi-level compositional optimization (MCO), and reinforcement learning (specifically, actor-critic methods).

Bilevel Optimization

Model Sparsity Can Simplify Machine Unlearning

1 code implementation NeurIPS 2023 Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, Sijia Liu

We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient.

Machine Unlearning Transfer Learning

What Is Missing in IRM Training and Evaluation? Challenges and Solutions

no code implementations4 Mar 2023 Yihua Zhang, Pranay Sharma, Parikshit Ram, Mingyi Hong, Kush Varshney, Sijia Liu

We propose a new IRM variant to address this limitation based on a novel viewpoint of ensemble IRM games as consensus-constrained bi-level optimization.

Out-of-Distribution Generalization

Federated Minimax Optimization with Client Heterogeneity

no code implementations8 Feb 2023 Pranay Sharma, Rohan Panda, Gauri Joshi

We analyze the convergence of the proposed algorithm for classes of nonconvex-concave and nonconvex-nonconcave functions and characterize the impact of heterogeneous client data, partial client participation, and heterogeneous local computations.

On the Convergence of Federated Averaging with Cyclic Client Participation

no code implementations6 Feb 2023 Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, Tong Zhang

Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL).

Federated Learning

FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning

no code implementations28 Jul 2022 Divyansh Jhunjhunwala, Pranay Sharma, Aushim Nagarkatti, Gauri Joshi

To remedy this, we propose FedVARP, a novel variance reduction algorithm applied at the server that eliminates error due to partial client participation.

Federated Learning

Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling

no code implementations21 Jun 2022 Sajad Khodadadian, Pranay Sharma, Gauri Joshi, Siva Theja Maguluri

To obtain these results, we show that federated TD and Q-learning are special cases of a general framework for federated stochastic approximation with Markovian noise, and we leverage this framework to provide a unified convergence analysis that applies to all the algorithms.

Q-Learning reinforcement-learning +1

Distributed Estimation in Large Scale Wireless Sensor Networks via a Two Step Group-based Approach

no code implementations17 Mar 2022 Shan Zhang, Pranay Sharma, Baocheng Geng, Pramod K. Varshney

To achieve greater sensor transmission and estimation efficiency, we propose a two step group-based collaborative distributed estimation scheme, where in the first step, sensors form dependence driven groups such that sensors in the same group are highly dependent, while sensors from different groups are independent, and perform a copula-based maximum a posteriori probability (MAP) estimation via intragroup collaboration.

Federated Minimax Optimization: Improved Convergence Analyses and Algorithms

no code implementations9 Mar 2022 Pranay Sharma, Rohan Panda, Gauri Joshi, Pramod K. Varshney

In this paper, we consider nonconvex minimax optimization, which is gaining prominence in many modern machine learning applications such as GANs.

Distributed Optimization Federated Learning

STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning

no code implementations NeurIPS 2021 Prashant Khanduri, Pranay Sharma, Haibo Yang, Mingyi Hong, Jia Liu, Ketan Rajawat, Pramod K. Varshney

Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achieve the desired solution.

Federated Learning

Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework

no code implementations21 Dec 2020 Pranay Sharma, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Xue Lin, Pramod K. Varshney

In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations.

Distributed Stochastic Non-Convex Optimization: Momentum-Based Variance Reduction

no code implementations1 May 2020 Prashant Khanduri, Pranay Sharma, Swatantra Kafle, Saikiran Bulusu, Ketan Rajawat, Pramod K. Varshney

In this work, we propose a distributed algorithm for stochastic non-convex optimization.

Optimization and Control Distributed, Parallel, and Cluster Computing

Why Interpretability in Machine Learning? An Answer Using Distributed Detection and Data Fusion Theory

no code implementations25 Jun 2018 Kush R. Varshney, Prashant Khanduri, Pranay Sharma, Shan Zhang, Pramod K. Varshney

Such arguments, however, fail to acknowledge that the overall decision-making system is composed of two entities: the learned model and a human who fuses together model outputs with his or her own information.

BIG-bench Machine Learning Decision Making

Cannot find the paper you are looking for? You can Submit a new open access paper.