Search Results for author:
Found 15 papers, 2 papers with code
High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise.
Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation
Stochastic approximation with multiple coupled sequences (MSA) has found broad applications in machine learning as it encompasses a rich class of problems including bilevel optimization (BLO), multi-level compositional optimization (MCO), and reinforcement learning (specifically, actor-critic methods).
Model Sparsity Can Simplify Machine Unlearning
We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient.
What Is Missing in IRM Training and Evaluation? Challenges and Solutions
We propose a new IRM variant to address this limitation based on a novel viewpoint of ensemble IRM games as consensus-constrained bi-level optimization.
Federated Minimax Optimization with Client Heterogeneity
We analyze the convergence of the proposed algorithm for classes of nonconvex-concave and nonconvex-nonconcave functions and characterize the impact of heterogeneous client data, partial client participation, and heterogeneous local computations.
On the Convergence of Federated Averaging with Cyclic Client Participation
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL).
FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learning
To remedy this, we propose FedVARP, a novel variance reduction algorithm applied at the server that eliminates error due to partial client participation.
Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling
To obtain these results, we show that federated TD and Q-learning are special cases of a general framework for federated stochastic approximation with Markovian noise, and we leverage this framework to provide a unified convergence analysis that applies to all the algorithms.
Distributed Estimation in Large Scale Wireless Sensor Networks via a Two Step Group-based Approach
To achieve greater sensor transmission and estimation efficiency, we propose a two step group-based collaborative distributed estimation scheme, where in the first step, sensors form dependence driven groups such that sensors in the same group are highly dependent, while sensors from different groups are independent, and perform a copula-based maximum a posteriori probability (MAP) estimation via intragroup collaboration.
Federated Minimax Optimization: Improved Convergence Analyses and Algorithms
In this paper, we consider nonconvex minimax optimization, which is gaining prominence in many modern machine learning applications such as GANs.
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning
Despite extensive research, for a generic non-convex FL problem, it is not clear, how to choose the WNs' and the server's update directions, the minibatch sizes, and the local update frequency, so that the WNs use the minimum number of samples and communication rounds to achieve the desired solution.
Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework
In this work, we focus on the study of stochastic zeroth-order (ZO) optimization which does not require first-order gradient information and uses only function evaluations.
Distributed Stochastic Non-Convex Optimization: Momentum-Based Variance Reduction
In this work, we propose a distributed algorithm for stochastic non-convex optimization.
Optimization and Control Distributed, Parallel, and Cluster Computing
Parallel Restarted SPIDER -- Communication Efficient Distributed Nonconvex Optimization with Optimal Computation Complexity
For online problems ($n$ unknown or infinite), we achieve the optimal IFO complexity $O(\epsilon^{-3/2})$.
Why Interpretability in Machine Learning? An Answer Using Distributed Detection and Data Fusion Theory
Such arguments, however, fail to acknowledge that the overall decision-making system is composed of two entities: the learned model and a human who fuses together model outputs with his or her own information.
