Search Results for author: Gauri Joshi

Found 45 papers, 13 papers with code

FedExP: Speeding Up Federated Averaging via Extrapolation

2 code implementations23 Jan 2023 Divyansh Jhunjhunwala, Shiqiang Wang, Gauri Joshi

Federated Averaging (FedAvg) remains the most popular algorithm for Federated Learning (FL) optimization due to its simple implementation, stateless nature, and privacy guarantees combined with secure aggregation.

Federated Learning

Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD

1 code implementation21 Feb 2020 Jianyu Wang, Hao Liang, Gauri Joshi

In this paper, we propose an algorithmic approach named Overlap-Local-SGD (and its momentum variant) to overlap the communication and computation so as to speedup the distributed training procedure.

MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling

4 code implementations23 May 2019 Jianyu Wang, Anit Kumar Sahu, Zhouyi Yang, Gauri Joshi, Soummya Kar

This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network.

Correlated Multi-armed Bandits with a Latent Random Source

2 code implementations17 Aug 2018 Samarth Gupta, Gauri Joshi, Osman Yağan

As a result, there are regimes where our algorithm achieves a $\mathcal{O}(1)$ regret as opposed to the typical logarithmic regret scaling of multi-armed bandit algorithms.

Multi-Armed Bandits

Accelerating Deep Learning by Focusing on the Biggest Losers

2 code implementations2 Oct 2019 Angela H. Jiang, Daniel L. -K. Wong, Giulio Zhou, David G. Andersen, Jeffrey Dean, Gregory R. Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C. Lipton, Padmanabhan Pillai

This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration.

Multi-Armed Bandits with Correlated Arms

2 code implementations6 Nov 2019 Samarth Gupta, Shreyas Chaudhari, Gauri Joshi, Osman Yağan

We consider a multi-armed bandit framework where the rewards obtained by pulling different arms are correlated.

Multi-Armed Bandits

Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization

1 code implementation NeurIPS 2020 Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, H. Vincent Poor

In federated optimization, heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client in each communication round.

Deep Kernels with Probabilistic Embeddings for Small-Data Learning

1 code implementation13 Oct 2019 Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, T. Yong-Jin Han

Experiments on a variety of datasets show that our approach outperforms the state-of-the-art in GP kernel learning in both supervised and semi-supervised settings.

Gaussian Processes Representation Learning +1

Probabilistic Neighbourhood Component Analysis: Sample Efficient Uncertainty Estimation in Deep Learning

1 code implementation18 Jul 2020 Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi, T. Yong-Jin Han

In this work, we show that the uncertainty estimation capability of state-of-the-art BNNs and Deep Ensemble models degrades significantly when the amount of training data is small.

COVID-19 Diagnosis Uncertainty Quantification

FedFisher: Leveraging Fisher Information for One-Shot Federated Learning

1 code implementation19 Mar 2024 Divyansh Jhunjhunwala, Shiqiang Wang, Gauri Joshi

Standard federated learning (FL) algorithms typically require multiple rounds of communication between the server and the clients, which has several drawbacks, including requiring constant network connectivity, repeated investment of computational resources, and susceptibility to privacy attacks.

Federated Learning

Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD

no code implementations3 Mar 2018 Sanghamitra Dutta, Gauri Joshi, Soumyadip Ghosh, Parijat Dube, Priya Nagpurkar

Distributed Stochastic Gradient Descent (SGD) when run in a synchronous manner, suffers from delays in waiting for the slowest learners (stragglers).

Active Distribution Learning from Indirect Samples

no code implementations16 Aug 2018 Samarth Gupta, Gauri Joshi, Osman Yağan

At each time step, we choose one of the possible $K$ functions, $g_1, \ldots, g_K$ and observe the corresponding sample $g_i(X)$.

Privacy Preserving

Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms

no code implementations22 Aug 2018 Jianyu Wang, Gauri Joshi

Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD.

A Unified Approach to Translate Classical Bandit Algorithms to the Structured Bandit Setting

no code implementations18 Oct 2018 Samarth Gupta, Shreyas Chaudhari, Subhojyoti Mukherjee, Gauri Joshi, Osman Yağan

We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter $\theta^*$.

Thompson Sampling

Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD

no code implementations19 Oct 2018 Jianyu Wang, Gauri Joshi

Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays.

Machine Learning on Volatile Instances

no code implementations12 Mar 2020 Xiaoxi Zhang, Jian-Yu Wang, Gauri Joshi, Carlee Joe-Wong

Due to the massive size of the neural network models and training datasets used in machine learning today, it is imperative to distribute stochastic gradient descent (SGD) by splitting up tasks such as gradient evaluation across multiple worker nodes.

BIG-bench Machine Learning

Slow and Stale Gradients Can Win the Race

no code implementations23 Mar 2020 Sanghamitra Dutta, Jianyu Wang, Gauri Joshi

Distributed Stochastic Gradient Descent (SGD) when run in a synchronous manner, suffers from delays in runtime as it waits for the slowest workers (stragglers).

Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

no code implementations3 Oct 2020 Yae Jee Cho, Jianyu Wang, Gauri Joshi

Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing.

Distributed Optimization Federated Learning +1

Bandit-based Communication-Efficient Client Selection Strategies for Federated Learning

no code implementations14 Dec 2020 Yae Jee Cho, Samarth Gupta, Gauri Joshi, Osman Yağan

Due to communication constraints and intermittent client availability in federated learning, only a subset of clients can participate in each training round.

Fairness Federated Learning

Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning

no code implementations8 Feb 2021 Divyansh Jhunjhunwala, Advait Gadhikar, Gauri Joshi, Yonina C. Eldar

Communication of model updates between client nodes and the central aggregating server is a major bottleneck in federated learning, especially in bandwidth-limited settings and high-dimensional models.

Federated Learning Quantization

Local Adaptivity in Federated Learning: Convergence and Consistency

no code implementations4 Jun 2021 Jianyu Wang, Zheng Xu, Zachary Garrett, Zachary Charles, Luyang Liu, Gauri Joshi

Popular optimization algorithms of FL use vanilla (stochastic) gradient descent for both local updates at clients and global updates at the aggregating server.

Federated Learning

Job Dispatching Policies for Queueing Systems with Unknown Service Rates

no code implementations8 Jun 2021 Tuhinangshu Choudhury, Gauri Joshi, Weina Wang, Sanjay Shakkottai

In multi-server queueing systems where there is no central queue holding all incoming jobs, job dispatching policies are used to assign incoming jobs to the queue at one of the servers.

Best-Arm Identification in Correlated Multi-Armed Bandits

no code implementations10 Sep 2021 Samarth Gupta, Gauri Joshi, Osman Yağan

In this paper we consider the problem of best-arm identification in multi-armed bandits in the fixed confidence setting, where the goal is to identify, with probability $1-\delta$ for some $\delta>0$, the arm with the highest mean reward in minimum possible samples from the set of arms $\mathcal{K}$.

Multi-Armed Bandits

Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer

no code implementations16 Sep 2021 Yae Jee Cho, Jianyu Wang, Tarun Chiruvolu, Gauri Joshi

Personalized federated learning (FL) aims to train model(s) that can perform well for individual clients that are highly data and system heterogeneous.

Personalized Federated Learning Transfer Learning

Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation

no code implementations NeurIPS 2021 Divyansh Jhunjhunwala, Ankur Mallick, Advait Gadhikar, Swanand Kadhe, Gauri Joshi

We study the problem of estimating at a central server the mean of a set of vectors distributed across several nodes (one vector per node).

Federated Learning

FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients

no code implementations28 Jan 2022 Jianyu Wang, Hang Qi, Ankit Singh Rawat, Sashank Reddi, Sagar Waghmare, Felix X. Yu, Gauri Joshi

In classical federated learning, the clients contribute to the overall training by communicating local updates for the underlying model on their private data to a coordinating server.

Federated Learning

Federated Minimax Optimization: Improved Convergence Analyses and Algorithms

no code implementations9 Mar 2022 Pranay Sharma, Rohan Panda, Gauri Joshi, Pramod K. Varshney

In this paper, we consider nonconvex minimax optimization, which is gaining prominence in many modern machine learning applications such as GANs.

Distributed Optimization Federated Learning

Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning

no code implementations27 Apr 2022 Yae Jee Cho, Andre Manoel, Gauri Joshi, Robert Sim, Dimitrios Dimitriadis

In this work, we propose a novel ensemble knowledge transfer method named Fed-ET in which small models (different in architecture) are trained on clients, and used to train a larger model at the server.

Ensemble Learning Federated Learning +1

Maximizing Global Model Appeal in Federated Learning

no code implementations30 May 2022 Yae Jee Cho, Divyansh Jhunjhunwala, Tian Li, Virginia Smith, Gauri Joshi

We provide convergence guarantees for MaxFL and show that MaxFL achieves a $22$-$40\%$ and $18$-$50\%$ test accuracy improvement for the training clients and unseen clients respectively, compared to a wide range of FL modeling approaches, including those that tackle data heterogeneity, aim to incentivize clients, and learn personalized or fair models.

Federated Learning

On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data

no code implementations9 Jun 2022 Jianyu Wang, Rudrajit Das, Gauri Joshi, Satyen Kale, Zheng Xu, Tong Zhang

Motivated by this observation, we propose a new quantity, average drift at optimum, to measure the effects of data heterogeneity, and explicitly use it to present a new theoretical analysis of FedAvg.

Federated Learning

Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling

no code implementations21 Jun 2022 Sajad Khodadadian, Pranay Sharma, Gauri Joshi, Siva Theja Maguluri

To obtain these results, we show that federated TD and Q-learning are special cases of a general framework for federated stochastic approximation with Markovian noise, and we leverage this framework to provide a unified convergence analysis that applies to all the algorithms.

Q-Learning reinforcement-learning +1

Multi-Model Federated Learning with Provable Guarantees

no code implementations9 Jul 2022 Neelkamal Bhuyan, Sharayu Moharir, Gauri Joshi

Federated Learning (FL) is a variant of distributed learning where edge devices collaborate to learn a model without sharing their data with the central server or each other.

Federated Learning

FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning

no code implementations28 Jul 2022 Divyansh Jhunjhunwala, Pranay Sharma, Aushim Nagarkatti, Gauri Joshi

To remedy this, we propose FedVARP, a novel variance reduction algorithm applied at the server that eliminates error due to partial client participation.

Federated Learning

On the Convergence of Federated Averaging with Cyclic Client Participation

no code implementations6 Feb 2023 Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, Tong Zhang

Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL).

Federated Learning

Federated Minimax Optimization with Client Heterogeneity

no code implementations8 Feb 2023 Pranay Sharma, Rohan Panda, Gauri Joshi

We analyze the convergence of the proposed algorithm for classes of nonconvex-concave and nonconvex-nonconcave functions and characterize the impact of heterogeneous client data, partial client participation, and heterogeneous local computations.

The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond

no code implementations18 May 2023 Jiin Woo, Gauri Joshi, Yuejie Chi

When the data used for reinforcement learning (RL) are collected by multiple agents in a distributed manner, federated versions of RL algorithms allow collaborative learning without the need for agents to share their local data.

Q-Learning Reinforcement Learning (RL)

Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels

no code implementations ICCV 2023 Yae Jee Cho, Gauri Joshi, Dimitrios Dimitriadis

For both cross-device and cross-silo settings, we show that FedLabel outperforms other semi-supervised FL baselines by $8$-$24\%$, and even outperforms standard fully supervised FL baselines ($100\%$ labeled data) with only $5$-$20\%$ of labeled data.

Federated Learning Pseudo Label

Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models

no code implementations12 Jan 2024 Yae Jee Cho, Luyang Liu, Zheng Xu, Aldi Fahrezi, Gauri Joshi

Foundation models (FMs) adapt well to specific domains or tasks with fine-tuning, and federated learning (FL) enables the potential for privacy-preserving fine-tuning of the FMs with on-device local data.

Federated Learning Privacy Preserving

Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems

no code implementations2 Feb 2024 Neharika Jali, Guannan Qu, Weina Wang, Gauri Joshi

Unlike homogeneous systems, a threshold policy, that routes jobs to the slow server(s) when the queue length exceeds a certain threshold, is known to be optimal for the one-fast-one-slow two-server system.

reinforcement-learning Reinforcement Learning (RL)

Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices

no code implementations8 Feb 2024 Jiin Woo, Laixi Shi, Gauri Joshi, Yuejie Chi

Our sample complexity analysis reveals that, with appropriately chosen parameters and synchronization schedules, FedLCB-Q achieves linear speedup in terms of the number of agents without requiring high-quality datasets at individual agents, as long as the local datasets collectively cover the state-action space visited by the optimal policy, highlighting the power of collaboration in the federated setting.

Federated Learning Offline RL +3

Cannot find the paper you are looking for? You can Submit a new open access paper.